Mobile user borne brain activity data and surrounding environment data correlation system

ABSTRACT

A mobile user borne brain activity data and surrounding environment data correlation system comprising a brain activity sensing subsystem, a recording subsystem, a measurement computer subsystem, a user sensing subsystem, a surrounding environment sensing subsystem, a correlation subsystem, a user portable electronic device, a non-transitory computer readable medium, and a computer processing device. The mobile user borne system collects and records brain activity data and surrounding environment data and statistically correlates and processes the data for communicating the data into a recipient biological, mechatronic, or bio-mechatronic system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This continuation application is related to and claims the benefit ofapplication Ser. No. 11/354,779 filed on 15 Feb. 2006 entitled “DynamicInteractive Region-Of-Interest Panoramic/Three-Dimensional ImmersiveCommunication System and Method” (abandoned); application Ser. No.11/830,637 filed on 30 Jul. 2007 entitled “Panoramic Image-Based VirtualReality/Telepresence Audio-Visual System and Method” (abandoned);application Ser. No. 12/266,308 filed on Nov. 6, 2008 entitled“Panoramic Adapter System and Method with Spherical Field-Of-ViewCoverage” (abandoned); U.S. patent application Ser. No. 13/507,190(granted as U.S. Pat. No. 9,101,279 B2) filed on 11 Jun. 2012 entitled“Mobile User Borne Brain Activity Data and Surrounding Environment DataCorrelation System”; U.S. patent application Ser. No. 13/294,986(granted as U.S. Pat. No. 9,344,612 B2) filed on 11 Nov. 2011 entitled“Non-Interference Field-Of-View Support Apparatus For A Panoramic FacialSensor”; U.S. patent application Ser. No. 14/788,437 (granted as U.S.Pat. No. 9,451,899 B2) filed on 30 Jun. 2015 entitled “Mobile User BorneBrain Activity Data and Surrounding Environment Data CorrelationSystem”; and U.S. patent application Ser. No. 15/152,214 (granted asU.S. Pat. No. 10,447,966 B2) filed on 11 May 2016 entitled“Non-Interference Field-Of-View Support Apparatus For A PanoramicSensor.” The above applications and patents are hereby incorporated byreference in their entireties into the present application.

This invention relates to data logging of neural correlates ofconsciousness derived from host internal physiological and externalpanoramic and positional sensors operated upon via computer processingto provide information for memory enhancement. The fields of neurology,biology, biometric sensor engineering, prosthetic devices, implants,augmented cognition, whole brain emulation, computer science,statistical analysis, fast fusion computer processing, panoramicimaging, surround audio, sub-vocalization, computer simulation,geospatial information, telecommunications, Internet search engines andsocial media, robotics, body modification, body worn, surgicallyimplanted, and body mounted devices are relevant to the presentinvention.

BACKGROUND OF THE INVENTION

A fundamental endeavor of mankind is to overcome natural and manmadelimitations through the use of invention and design; with two innategoals being to live a contented life and in due course overcomemortality. The present invention is especially aimed at furthering thesegoals by the disclosure and use of a data logging and memory enhancementsystem and method. While humans have the natural ability to pass onphysical attributes via genetics directly through reproduction, humansdo not naturally have the ability to pass memory and thought processesthrough reproduction. And only very recently in human history hasmankind had the ability to not just evolve but to determine how itevolves. Traditionally, environment caused changes in man's evolution,often taking many generations and thousands to millions of years forsignificant changes to naturally occur. But increasingly, humans can usemodern procedures and technological advances to instantly change theirmake-up. Increasingly, modern technology is being offered that goesbeyond traditional maintenance of the human faculties that we are bornwith, and is providing systems and methods to substitute, replace, orenhance what humans are provided with naturally at birth. The artificialheart, the Dobelle Eye, growing artificial body parts, using stem cellsto differentiate into host cells, and genetic engineering are just a fewexamples. This invention is aimed at providing “designed evolutionary”systems and methods that accomplish this type of utility. Part of beinga sentient being is being self-aware and realizing that there is a past,present, and future consequences of one's actions. It is thereforeconceived as part of this invention that the user of the data loggingand memory enhancement system, when coupled with problem solving,mobility, and available resources, will perform maintenance that willallow himself, herself, or itself to continue to exist in some fashionindefinitely.

The brain is the center of all human thought and memory. The sentienceof a being ends up encompassing what the being processed and retained inthe being's central nervous system. The being is constantly perceivingthe environment that surrounds in order to update his or her thought andmemory. With this in mind, a central objective of the present inventionis developing a human internal to external correlation of consciouspercepts relative to a particular being realized in a system and methodfor personnel data logging and memory enhancement. For purposes of thepresent invention, the concept of “Neural Correlates of Consciousness”(NCC) and “Conscious Percept” (CP) are discussed and defined inpublications and presentations by, for example, Christof Koch in “TheQuest for Consciousness: A Neurobiological Approach” dated 2010 and asdepicted in a diagram herein modified in accordance with the presentinvention, entitled “Neural Correlates of Consciousness” dated 2008 [1].Neural Correlates of Consciousness (NCC) can be defined as “the minimalneuronal mechanisms jointly sufficient for any one conscious percept” ofa self-aware being, the being also aware of the surrounding environment,and other conscious contents to which that being is relating at a giventime. A “conscious percept” may be defined as a subject's focus ofattention as a being, machine, or bio-mechanical system in a particularsurrounding environment at a given time. A surrounding environment mayalso be referred to as “place”, which may, for example, be defined byimagery, audio, brain activity, or geospatial information. Achievingthis objective can be done by recording and measuring internal andexternal activity and relating the activity in the mind to the activityand subjects that the person is thinking about in the surroundingenvironment. Studies have taught us that various senses can stimulatethe central nervous center. Examples focused on in the present inventionare those which yield the most utility for learning. Approximately 78%of all information taken in is through our eyes, 12% through our ears,5% through touch, 2.5% through smell, and 2.5% through taste. It is anobjective of the present invention, and it will be understood to thoseskilled in the art, that various internal and external types of sensorsystems (i.e. audio, imagery, video camera, geospatial, position,orientation, brain activity, and biometric systems) may be used torecord sensory data and that this data may be processed in a computer tobuild a correlation, transcription, and translation system for human tomachine interaction. Statistical correlations are useful in the presentinvention because they can indicate a predictive relationship that canbe exploited in practice. A computer can operate upon recorded sensorydata using adaptive filters (i.e. Kalaman and/or Bloom filter algorithmsimplemented in computer language) to determine the correlation betweenthe internal and external representations to determine the strength ofthe statistical relationship between internal and externalrepresentations. Thresholds for retaining, disregarding, or acting uponthe data may be based on the statistical relationships and used todetermine targeted data output. In the present invention, translation isthe communication of the meaning of a source-language, be it human ormachine. It is an objective of the present invention to incorporatemachine translation (MT) as a process wherein computer program(s)analyze inter-related raw and preprocessed sensor data and producetarget output data (i.e. human understandable GUI text, video orsynthesized voice audio output into human interactive input devices of ahuman user) with little or no human intervention. In the context of thecurrent invention, computer-assisted translation (CAT), also called“computer-aided translation,” “machine-aided human translation” (MAHT),and “interactive translation,” is a form of translation wherein amachine translation system uses machine language to create a targetlanguage, be it human or machine, correlated with text,sub-vocalization, brain activity, and sensory signatures of subjects andactivities in the surrounding environment with the assistance ofcomputer program(s). It is an objective of the present invention to usethe above translations to form the basis of a relational database whichmay be drawn upon by a user to perform various functions using a mobilecomputing device such as a smartphone or the like as described herein.

Spherical field-of-view sensing and logging about the user is preferablewhen it comes to recording how the mind works because the mindconstantly perceives the space one finds himself or herself occupying.In an academic paper entitled “Intelligent Systems in the Context ofSurrounding Environment” Joseph Wakeling and Per Bak of the Departmentof Mathematics, London, UK, dated 29 Jun. 2001, describe a biologicallearning pattern based on “Darwinian selection” that suggests thatintelligence can only be measured in the context of the surroundingenvironment of the organism studied: i.e. “We must always consider theembodiment of any intelligent system [2]. The preferred embodimentreflects that the mind and its surrounding environment (including thephysical body of the individual) are inseparable and that intelligenceonly exists in the context of its surrounding environment.” Studies byO'Keefe, J. and Nadel, L. (1978) entitled “The Hippocampus as aCognitive Map”, Clarendon Press: Oxford [3] and Rotenberg, A., Mayford,M., Hawkins, R. D., Kandel, E. R., and Muller, R. U. (1996) [4] andclassic studies of John O'Keefe and John Dostrovsky (1998) [5], point tostrong evidence why a video logging machine needs to provide a panoramicfield-of-view FOV about a user in order to get a true representation orreproduction of their consciousness In 1971 it was discovered that thepyramidal cells of the hippocampus—the cells one examines artificiallyusing electrical stimuli to the Schaffer collateral pathway whilestudying LTP—are “place cells”; they actually encode extra-personalspace in real life. A given pyramidal cell will fire only when the headof a user is in a certain part of an enclosed space: the cell's placefield. Thus, when a person walks borne with the present invention in agiven space, a particular subset of pyramidal cells in the hippocampusbecomes active. When the user is in different space, different sets ofpyramidal cells become active. Cells of the hippocampus form an internalneural representation, or “cognitive map” of the space surrounding theuser. This holistic neural representation permits the user to solvespatial problems efficiently. And when placed in a new environment, aperson forms an internal representation of the new space (thecoordinated firing of a population of place cells) within minutes, andonce this representation is formed it is normally stable for at leastseveral days. The same cell will have the same firing field each timethe person is reintroduced to that environment. When now placed in asecond environment, a new map is formed—again in minutes—in part fromsome of the cells that made up the map of the first environment and inpart from pyramidal cells that had been silent previously. These placecells and spatial memory can be studied by recording brain patternactivation using MRI, and various other brain activity systems 40 suchas AMR, fMRI, fNRI, EEG, PET, or DECI to record brain activity fromindividual pyramidal cells in the hippocampus (ref. Kandel and Squire,1998) [6]. Studies show that regions of the brain that have place cellsthat are active when one is in a familiar place versus when one is notin a familiar place. Activity is especially noticeable in these cellswhen a person is navigating a space in the dark. Human memory works torecall and visualize what was there in the daylight to help a user ofthe present invention navigate a dark space.

Neurological research has identified specific locations, processes, andinteractions down to the human neuron and molecular level for thinkingand memory. Research has shown that human neurons and synapse both areactively involved in thought and memory, and that brain imagingtechnology such as Magnetic Resonance Imaging (MRI), Nuclear MagneticResonance Imaging, or Magnetic Resonance Tomography (MRT) can be used toobserve this brain activity at the molecular level. Recently atomicmagnetometers have begun development of cheap and portable MRIinstruments without large magnets used in traditional MM machines toimage parts of the human anatomy, including the brain. There are over100 billion brain cells/neurons in the brain, each of which has synapsesthat are involved in memory and learning, which can also be observed bybrain imaging techniques. It has also been proven that new brain cellsare created whenever one learns something new. Whenever stimuli in theenvironment or through thought make a significant enough impact on thebeing's brain, new neurons are formed. During this process, synapsescarry on electro-chemical activities that reflect activity related toboth memory and thought. Using modern technological devices, such as anAtomic Magnetometer, this activity in the brain at the molecular levelcan be detected, measured, stored, and operated upon using computersaccording to the present invention as these processes are taking placein the brain. Research has also shown that, even though there areimportant similarities in the brain activity of different people, eachperson has a unique brain “fingerprint.” This fingerprint of the brainis unique to each person's thought processes and how and where theystore their memories in their brain. It is an objective of the presentinvention to facilitate recording and translating the uniqueness of asubject's brain and the subjects corresponding brain activity. It is anadditional objective of the present invention to provide a universalbrain translation system and method that facilitates communicationbetween different beings, machines, or a combination thereof.

In September 2006 Stefan Posse and his colleagues at the University ofNew Mexico used MM techniques to observe brain activity correlated withthe thought of a single word. And they recently recorded longer imagingsequences and decomposed the thought processes into individual thoughts[7]. When images of Marilyn Monroe were shown, a specific neuron fired,when images of another actor were shown, a neuron specific to that actorfired. Likewise, Francis Krick and Christof Koch in the periodicalNature Neuroscience, Vol. 6, number 2, dated February 2003, in anarticle entitled “A Framework for Consciousness” along with their morerecent findings demonstrate that certain neurons fire selectively tocertain visual stimuli [8]. Koch argues for including the neuralcorrelates for conscious percepts as any part of understanding how humanbeings are consciously aware. Koch research has shown that neuralcorrelates of both basal arousal and activity in the inferior temporalcortex are necessary for a human being to be consciously aware. And thatbrain decoding techniques can be translated into images based on readinga patient's mind. In a study, 20-30 specific neurons were listened to inorder to infer what the patient was conscious of Research by Koch hasalso shown that physical input (i.e. A person actually looking at anobject) and imagined input (i.e. A person closing their eyes andimagining an object in their mind) stimulated the same neurons. It is anobject of the present invention to correlate repeated recordings andloggings of user physiological activity (i.e. user brain activity,sub-vocal imitations, etc.) with recordings and loggings of thesurrounding environmental activity (i.e. panoramic video images of gazeof the user upon a subject, etc.) to build an esemplastic patternedlanguage using the present invention. The computerized logging andassistance system that forms the present invention thus yielding arepresentation of the consciousness and understanding of the world fromthe given point-of-view of the being whose information is operated upon.And the computerized logging and assistance system that forms thepresent invention thus providing an informational system that may beoperated upon to assist a user being, machine, or combination thereof innegotiating the world in which he, she, or it respectively lives oroperates.

An example of a brain activity sensing system, providing enablingtechnology incorporated into the present invention, is a portableMagnetic Resonance Imaging device such as one according to the AtomicMagnetometer Sensor Array Magnetic Resonance (AMR) Imaging Systems andMethods. Recently, portable Atomic MR systems, such as those describedin U.S. Patent Publication 2009/0149736, dated 11 Jun. 2009 by Skidmoreet al. and U.S. Patent 2010/0090697, dated 15 Apr. 2010 by Savukov, havebeen disclosed that are of a type compatible with and enabling of thepresent invention. Further, John Kitching, a physicist at the NationalInstitute of Standards and Technology in Boulder, Colo. has developedtiny (grain of rice size) atomic magnetic sensors of a type compatiblefor use in the present invention [37]. Specifically, systems and devicesdisclosed in the Skidmore patent and Kitching presents a wearableportable array, of reduced size, low power consumption, reducible to awafer-level, has rapid signal transfer, and with decreased magneticfield that facilitates lower cost and easy mounting on and/or inside aperson, animal, or inanimate object. U.S. Patent Application20100016752, by Jeffery M. Sieracki dated 21 Jan. 2010 entitled Systemand Method for Neurological Activity Signature Determination,Discrimination, and Detection discloses a system for automaticallycorrelating neurological activity to a predetermined physiologicalresponse comprising: at least one sensor operable to sense signalsindicative of the neurological activity; a processing engine coupled tosaid sensor, said processing engine being operable in a first systemmode to execute a simultaneous sparse approximation jointly upon a groupof signals sensed by said sensor to generate signature informationcorresponding to the predetermined physiological response; and, adetector coupled to said sensors, said detector being operable in asecond system mode to monitor the sensed signals and generate uponselective detection according to said signature information a controlsignal for actuating a control action according to the predeterminedphysiological response.

Still alternatively, U.S. Patent Application 2010/0042011, dated 18 Feb.2010, by Doidge et al. entitled “Three-dimensional Localization,Display, Recording, and Analysis of Electrical Activity in the CerebralCortex” discloses a computerized Dynamic Electro-cortical Imaging (DECI)method and apparatus for measuring EEG signatures of the brain in realtime. The DECI system and method is portable and can be worn by the userto generate dynamic three-dimensional (voxel) information of theelectrical activity occurring in the cerebral cortex of the brain. TheDECI system is of a type that may be incorporated in the presentinvention to provide brain activity information according to the presentinvention. U.S. Patent Application 2010/0041962, dated 18 Feb. 2010 byCausevic et al., entitled “Flexible Headset for Sensing ElectricalActivity” discloses a headset worn on the outside of the head forsensing brain activity.

Additionally, scientific studies show that images we recall in ourimagination are not always as detailed as a photographic image. In 1999,researchers led by Yang Dan at University of California, Berkeleydecoded neuronal firings to reproduce images seen by laboratory animals[36]. The team used an array of electrodes embedded in the thalamus(which integrates all of the brain's sensory input) of animals.Researchers targeted 177 brain cells in the thalamus lateral geniculatenucleus area, which decodes signals from the retina. The animals wereshown eight short movies, and their neuron firings were recorded. Usingmathematical filters, the researchers decoded the signals to generatemovies of what the animals saw and were able to reconstruct recognizablescenes and moving objects. An object of the present invention is toprovide imagery and audio of the subject of the CP and surroundingenvironment that is correlated to brain activity which can be queried bya user of the invention from logged information recorded by theinvention which is more complete and accurate than what the brainremembers. To derive this utility from the above-mentioned brainactivity systems, like the AMR system, the resulting brain activitysignatures are related to a thoughts and memories as associated withthings in the surrounding environment with respect to the individualusing the AMR system. A monocular or binocular camera system may beincorporated into the present invention. But preferably, a camera systemwith stereoscopic capability is incorporated. U.S. Patent Application20070124292 A1, by Kirshenbaum et al., dated 31 May 2007, entitledAutobiographical and Other Data Collection System describes a system forcollecting/recording, storing, retrieving, and transmitting videoinformation that may be incorporated into the present invention.Stereoscopic cameras that approximate human vision are preferablebecause they reflect how humans naturally see and experience the worldand provide depth clues to the brain. Panoramic stereoscopic cameras arealso more preferable because they provide more measurable data, addedspatial awareness like that what person's experience, and allow thereplay of the total surrounding environment is more attune to what isactually stimulating the user's senses, memories, and resulting thoughtsin the real world. Portable head-mounted panoramic video cameras of atype that may be used in the present invention include U.S. Pat. No.6,552,744 B2 by Chen, dated Apr. 22, 2003, entitled Virtual RealityCamera which presents a camera which records discrete still or videoimages that can be stitched together to create a panoramic scene thatincorporates computer processing so that the user may pan and zoomaround the panoramic scene; U.S. Patent Application 2001/00105555 andU.S. Pat. No. 6,539,547, by Driscoll, dated Aug. 2, 2001, discloses aMethod and Apparatus for electronically recording, storing, anddistributing panoramic images from a panoptic camera system to a remotelocation using the Internet; U.S. Patent Publication 2005/0157166 byPeleg, dated Jul. 21, 2005 entitled Digitally Enhanced Depth Image whichdiscloses a camera method to simultaneously record, store, and processpanoramic stereoscopic imagery; U.S. Pat. No. 5,023,725, by McCutchen,dated Jun. 11, 1991, FIG. 21, which discloses a cap with a plurality ofhigh-resolution video cameras that record a plurality of imagery thatmay be stitched together to form a hemispherical scene; U.S. Patent20020015047 Okada, Hiroshi” et al., dated Feb. 7, 2002 entitled “Imagecut-away/display system” that describes a panoramic camera, processing,display system in which the images are combined for forming a singlewide-area view image for use as a virtual environment, telepresenceenvironment, texture mapped three-dimensional simulated environment, oran augmented reality environment consistent for use in the presentinvention; U.S. Patent Application Publication 2005/0128286 dated 16Jun. 2005 by Angus Richards that discloses a panoramic camera mountedhelmet that also includes a head-mounted display (HMD) withtelecommunication capabilities; U.S. Pat. Nos. 5,130,794, and 5,495,576,and grandparent, parent, and pending related applications by Ritchey etal.; and U.S. Patent Applications 2005/0128286 dated 16 Jun. 2006 and2006/0082643 dated 20 Apr. 2006 that disclose HMD systems of a typecompatible for incorporation in the present invention. All of the camerasystems cited in this paragraph are of a type that may be incorporatedas a component of the present invention.

Still alternatively, eye-in and eye-on contact lenses may includecameras for recording and displaying imagery according to the presentinvention. For example, a camera device that is mounted on and/or insidethe eye is disclosed in US Patent 20090189974 A1, by Michael F. Deering,dated 30 Jul. 2009, entitled Systems Using Eye Mounted Displays (EMD).Deering describes a still and/or video camera could be placed directlyon the eye mounted display worn on or in the user's eye(s). Such acamera, in essence, automatically tracks the motions of the user'seye(s) because it is effectively part of the user's eye(s). The eyemounted camera is folded within the EMD using some of the same opticalfolding techniques used in folding the display optics of the EMD. Theprocessing of the image is handled on the contact lens, an electronicspackage on the user's body, or by a remote processing center. A remoteuser can pan and tilt the camera to point in the same direction as theuser's eyes, using the direction information from the eye trackingsubsystem. Such a camera greatly reduces the time and physical grabbingof an external camera when taking a picture; as an example, aparticularly gorgeous sunset can be photographed with something assimple as a quick glance and a double eye blink. The camera can belocated in one or both eyes. A plurality of camera systems, like EMD andpanoramic camera systems, may be integrated in the present invention toattain the required FOV coverage and overall system functionality. AnEMD system of this type may provide capture and/or display for thepresent invention, and may transmit to and from the smartphone whenincorporated according to the present invention. Additionally, anotherEMD design consists of a contact lens that harvests radio waves to poweran LED that displays information beamed to the contact lens from mobiledevices, like a smartphone. The EMD system was invented by Babak Parvizand is currently in prototype at the University of Washington (Ref. NewScientist, 12 Nov. 2009 by Vijaysree Venkatraman) [9]. The above systemsare of a type compatible with and are incorporated into the presentinvention.

A smartphone is a portable electronic device (PED) that combines thefunctions of a personal digital assistant (PDA) with a mobile phone.Smartphones typically have computer and computer processing hardware,firmware, and software built in to the unit. An example of a smartphoneis the iPhone 4S and 5, sold by Apple Inc. Later models added thefunctionality of portable media players, low-end compact digitalcameras, pocket video cameras, and global positioning system (GPS)navigation units to form one multi-use device. Modern smartphones alsoinclude high-resolution touch screens and web browsers that displaystandard web pages as well as mobile-optimized sites. High-speed dataaccess is provided by Wi-Fi and Mobile Broadband. The most common mobileoperating systems (OS) used by modern smartphones include Google LLC'sAndroid® OS, Apple Inc.'s iOS™ OS, Nokia Corporation's Symbian™ OS,Research In Motion Limited's or Blackberry Limited's BlackBerry® OS,Samsung Electronics Co., Ltd.'s SAMSUNG BADA OS® OS, MicrosoftCorporation's WINDOWS PHONE OS, Hewlett-Packard Development Company's orLG Electronics Inc.'s webOS® OS, and embedded Linux distributions suchas Maemo™ and MeeGo®. Such operating systems can be installed on manydifferent phone models, and typically each device can receive multipleOS software updates over its lifetime.

It is also known in the art that small independent pill capsules may beused to capture imagery. A very small wireless video camera and lens,transceiver, data processor and power system and components that may beintegrated and adapted to form the panoramic capable wirelesscommunication terminals/units is disclosed by Dr. David Cumming ofGlasgow University and by Dr. Blair Lewis of Mt Sinai Hospital in NewYork. It is known as the “Given Diagnostic Imaging System” andadministered orally as a pill/capsule that can pass through the body andis used for diagnostic purposes. U.S. Pat. No. 7,662,093, by Gilad etal., dated 16 Feb. 2010, entitled Reduced Size Imaging Device describesa swallowable imaging capsule that includes an imager, processing, andwireless transmission system that may be incorporated and is compatiblewith the present invention. Others similarly include U.S. Pat. No.7,664,174 and U.S. Patent Application 20080033274 and 20080030573. Smallpen cameras, tie cameras, and so on used in the spy and surveillance mayalso be incorporated into forming camera components of the presentinvention. Objective micro-lenses suitable for taking lenses in thepresent invention, especially the panoramic taking assembly, aremanufactured and of a type by AEI North America, of Skaneateles, N.Y.,that provide alternative small and compact visual inspection systems.AEI sales micro-lenses for use in borescopes, fiberscopes, andendoscopes. AEI manufacture objective lens systems (including theobjective lens and relay lens group) from 4-14 millimeters in diameter,and 4-14 millimeters in length, with circular FOV coverage from 20 toapproximately 180 degrees. Of specific note is that AEI can provide anobjective lens with over 180 FOV coverage required for some embodimentsof the panoramic sensor assembly like that incorporated in the presentinvention required in order to achieve overlapping adjacenthemispherical FOV coverage of two back-to-back fisheye lenses orstereoscopic panoramic coverage when four lenses are incorporated at90-degree intervals. The above cameras, transmitters, and lenses may beincorporated into the above video logging system or other portion of thepanoramic capable wireless communication terminals/units to form thepresent invention. Camera systems may be operated by powered andcontrolled via wire clad, fiber-optics, or over a radio frequencysignal. Camera signals may be processed and transmitted separately ormultiplexed by any manner familiar to those in the art in the presentinvention. Both EMD and pill camera technology are enabling and areincorporated in the present invention to record and transmit imagery ofthe user and the scene surrounding the user in the present invention.

As stated above, deriving utility from the above-mentioned brainactivity systems includes relating the brain activity to a subject(s) inthe surrounding environment at the time that the focus was on thesubject observed. User born position orientation, geospatial positionand orientation systems, target designators, and eye tracking systemsmay be incorporated in the present invention to accomplish the task ofrecording what the attention of the user is focused upon. Pointingdevices may be any user-operated pointing device including, but notlimited to, a joystick, a trackball, a touch-sensitive pad or screen, aset of directional “arrow” cursor control keys, a helmet-mounted sight,or an eye-tracking system. Many navigation systems, surveillance systemsand weapon systems provide a user with a video image of a region ofinterest (ROI) from which the user may designate an object or featurefor tracking. In a typical tracker, the user selects the desired targetand from that point onward the target is tracked automatically. Knowntechniques for video-based target designation employ a use operatedpointing device (e.g., joystick, trackball, helmet-mounted sight,eye-tracking system. etc.) to either move a cursor/marker or move agimbal on which the camera is mounted so that a marker (e.g. acrosshair) is located on the desired target on the live video display.Then, by pushing a button, the user finally locks the tracker on thecurrent target. A video scalar and rangefinder may be incorporated aspart of the target tracking system. A tracking module is then actuatedand attempts to reliably acquire a trackable target at the designatedposition within the image for subsequent automated tracking. Targettracking systems may be integrated with eye tracking systems todetermine what the eyes of a person are focused upon. Tracking andpointing devices may be manually operated, or automatically operated bya computer given a rule set. Eye tracking systems are known in prior artthat monitor the position of a user's eye within its socket in order todetermine the user's line of gaze; for example, to enable the user tocontrol a device, such as a weapon, by eye movements, or to determinewhether the user is watching a predetermined location, such as alocation on a television screen, or simply to determine the state ofwakefulness of the user.

Furthermore a number of different methods have been proposed formonitoring the position of the user's eye associated with gaze and focuson a subject in the user's field-of-view (FOV), including the so-calledcorneal reflection (CR) method in which a point light source is used toproduce a bright image on the anterior surface of the cornea, and atracking system monitors the position of the image. A differentialCR/pupil tracking method has been developed in which the relativepositions of the pupil and a corneal reflection are monitored by asuitable camera and a wavelength-sensitive beam splitter being used toensure that the user's view is not obstructed by the light source andcamera. This method is less sensitive to sensor movements. Generally,the eye is illuminated by a near infrared source (or multiple sources)and a solid-state video camera captures an image of the eye. Inso-called bright pupil imaging the light source produces a light beamwhich is coaxial with the camera axis, and light reflected back from theretina making the pupil appear to be a bright circle, the apparentbrightness increasing roughly with the fourth power of pupil diameter.In so-called dark pupil imaging the light source produces a light beamwhich is off axis relative to the camera axis, and a dark pupil image isproduced. Real time image analysis is used to identify the pupil andcorneal reflections and to find their centers. Portable target trackingand pointing devices of a type that can be incorporated in the presentinvention to associate the image observed in the surrounding environmentwith specific subjects there-in and brain activity to facilitaterecording correlate designation include the eye tracking systemgenerally described above and specifically described in U.S. PatentApplication 20040196433, by Durnell, dated 7 Oct. 2004, titled EyeTracking System, and in U.S. Patent Application 20080205700, by Nir,dated 28 Aug. 2008 titled Apparatus and Method for Assisted TargetDesignation which includes video designation and tracking via imageryand/or directional audio. The above systems referenced in this paragraphproduced information that can be digitally stored and processed by acomputer. The eye tracking, gaze, and directional FOV, and GPS derivedfrom systems described in this paragraph can be correlated with recordedand stored AMR, and camera data of objects and scenes according to thepresent invention. The Ultra-Vis, Leader, system developed by ARA, whichincludes the subsidiary companies MWD, Vertek, and KAD, Lockheed Martin,and Microvision Incorporated is a type of target designation andtracking system that may be integrated into the present invention [32,33]. The portable iLeader system includes a HMD system with amicro-laser range finder system for target designation, see througheyewear, head 204 and eye 205 tracking system, waveguide displaygoggles, video cameras for recording the view the user is seeingdirectly ahead of where he is looking, helmet electronics, eye trackingand target designation system, voice mics and earbuds, and an associatedelectronics unit to control the HMD, telecommunications network and GPSinterface, iGlove, battery power and sensor feed, and a soldieraugmented reality (AR) system. In the planning and patrol mode view, theuser's see-through HMD of the iLeader system is operated by the user todesignate and record targets in the surrounding environment and overlayinformation on a see-through display. The overlaid information displayedto the user may be from associated sensors the user is wearing, sensorsother users are wearing, or from other information on networked devicesthat is wirelessly transmitted from a remote location that is part ofthe telecommunication system and network that includes the iLeadersystem. Technology of a type disclosed in the iLeader system isconsistent with and may be incorporated into the present invention.

As mentioned above, audio input systems provide a significant portion ofhuman sensory input. A microphone system is incorporated to record audiofrom and about the user as part of the video logging system described inthe present invention. Microphones are faced inward to record audio fromthe user and outward to record audio about the user. Typically,microphones are located on the user as a device worn or carried by theuser. Small microphones are known to those in the art and are commonlyused in the hand-free cell phone operation and known as throat mics thatfit around the ear or as lapel mics worn by those in the televisionindustry and security industry and are of a type that is compatible withand incorporated into the present invention. The microphone can be partof an audio recording or communication system common on cellulartelephones and in the cellular telephone industry. Alternatively, athree-dimensional surround sound ambisonic audio recording system existto capture using a tetrahedrally arranged quartet of cardioid patternmicrophone capsules connected to some simple circuitry to convert theoutputs to a standard B-format signal. B-format signals represent a 3Dsound-field with four signals; X, Y and Z representing three orthogonalfigure of eight patterns and an omni-directional W reference signal.Audio from ambisonic microphones may be spatially encoded using surroundsound encoders to output spatial audio may be played back in a user'searphones or earbuds. Ambisonic microphones may be distributed in anoutward facing manner according to the present invention. Ambisonichardware known as TetraMic Spheround with associated software of a typeapplicable to the present invention is manufactured by Core Sound ofTeaneck, N.J., USA [34].

Vocal representations of the user or from a remote user, be they wordsspoken aloud or sub-vocalized, may be sensed and provide data inputaccording to the present invention. Audio can be used for correlationpurposes or for command and control of the logging and enhancementsystem according to the present invention. Speech recognition (alsoknown as automatic speech recognition or computer speech recognition)converts spoken words to text. The term “voice recognition” is sometimesused to refer to recognition systems that must be trained to aparticular speaker—as is the case for most desktop recognition software.Recognizing the speaker can simplify the task of translating speech. Inthe present invention a microphone is a user interface for recordingaudio signatures of the user and surrounding environment for input in toan associated computer in order to facilitate hands-free computing.Conventional voice-command systems that use conventional voicerecognition systems of a type that may be used in the present inventioninclude the Kurzweil Applied Intelligence (KAI) Speech RecognitionSystem for commercial use [35]. In the present invention, a microphoneis a user interface for recording audio signatures of the user andsurrounding environment for input into an associated computer in orderto facilitate hands-free computing.

An embodiment and sensor input component of the present inventionincludes a sub-vocalization system. Sub-vocalization is the tendency ofa user to silently say individual words to themselves as they read orthink. Sub-vocal recognition (SVR) is the process of takingsub-vocalization and converting the detected results to a digitaltext-based or text-synthesized voice audio output. It is similar tovoice recognition except it is silent sub-vocalization being detected. Asub-vocalization system of a type that may be incorporated into thepresent invention as a component disclosed in U.S. Pat. No. 6,272,466,dated 7 Aug. 2001, by Harada, et al., entitled “Speech detectionapparatus using specularly reflected light” and that described in theongoing NASA Sub-vocal Recognition (SVR) program began in 1999, andlater renamed the Extension of Human Senses program. In the NASA programmuscles of the vocal tract (e.g. electromyographic or EMG) signaturesare sensed by contact sensors placed on the throat (either internally orexternally to the body) [38]. The signatures are read out as electricalsignals which are translated by a computer into patterns recognized byclassifiers as word or word components. Another sensor input system thatmay be integrated with the present logging and memory enhancement systemand method include infrared and LIDAR systems. LIDAR (Light Detectionand Ranging) is an optical remote sensing technology that measuresproperties of scattered light to find range and/or other information ofa distant target. LIDAR systems can see through fog and darkness torecord the shape and motion of objects in their FOV, overcoming thelimitation of visible spectrum cameras. LIDAR systems and methods of atype that may be integrated into and is compatible with the presentinvention are those found in U.S. Patent Application 2003/0154010 andU.S. Pat. No. 6,859,705, by Rae et al., dated 14 Aug. 2003 and 22 Feb.2005, entitled “Method for Operating a pre-crash sensing system in avehicle having a countermeasure system” using a radar and camera; U.S.Patent 2007/0001822 by Karsten Haug, dated 4 Jan. 2004, entitled “Methodfor improving vision in a motor vehicle”; and that mentioned in U.S.patent application Ser. No. 11/432,568 entitled “Volumetric PanoramicSensor Systems” filed May 11, 2006 and LIDAR systems cited in relatedpatent applications by the present inventor. An objective of the presentinvention is to provide and embodiment to the present invention whichincludes a LIDAR system for logging the surrounding environment: and manportable systems described in U.S Patent Application Publication2011/0273451, dated 10 Nov. 2011, by Salemann; and a publicationentitled “An approach for collection of geo-specific 3D features fromterrestrial LIDAR”, by Dr. David Optiz et al., of Overwatch GeospatialIncorporated, of Missoula, Mont., dated 28 Apr. 2008, at the ASPRSConference [39].

Turning now to user feedback systems of a type incorporated into thepresent invention. Feedback to the user can be through any of the user'ssenses. Portable audio-visual devices of a type that may be incorporatedin the present invention to provide visual and audio information to theuser include information appliances like cellular phones, head-mounteddisplays, laptops, and speaker headphones. Additionally, separate eyeand audio capture and presentation devices may be worn by the user. Theseparate devices may be connected via radio-frequency, infrared, wire,fiber-optic communications network on or off the user. Processing of theaudio and visual signature information may be at the site of the sensoror downstream in the body, or outside the body on a system mounted on,carried by, or at a remote server in communication with the user's videologging and enhancement/assistance system.

According to many users, a current limitation of panoramic head-mounteddisplay (HMD) systems integrated with panoramic camera systems is thatthey are too heavy and bulky. The additions of wider field-of-viewdisplays and viewing optics, microphones, speakers, cameras, globalpositioning systems, head and eye tracking systems, telecommunication,associated power and processing capabilities, along with helmet paddingcan add additional weight and bulkiness. These problems contribute tothe majority of head-mounted displays being too large and not beingportable. Correspondingly, a limitation is that putting-on, adjusting,and taking-off the HMD is a difficult task. Finally, another limitationis that good head-mounted displays are expensive. Head-mounted display(HMD) devices of a type that are compatible with the present inventionare described in the present inventors previous disclosed prior art. HMDdesign well known to those skilled in the art and that may be used inthe present invention is described in the following papers: Head-WornDisplays, The Future Through New Eyes, by Jannick Rolland and OzanCakmakci, published by the Optical Society of America, April 2009 [10];Head-Worn Displays: A review by Jannick Rolland and Ozan Cakmakci,published IEEE in the Journal of Display Technology, Vol. 2, No. 3,September 2006 [11]. Specifically, a type of system applicable to thepresent invention is a low profile writeable holographic head worndisplay (HWD) that has see-through capabilities that facilitateaugmented reality. U.S. Patent Application 20100149073, by David Chaumet al., dated 17 Jun. 2010, entitled “Near to Eye Display System andAppliance” is such a holographic type of display compatible with andthat is incorporated into the present invention. Such a systemcompatible with and integrated by reference into the present inventionmanufactured by Microvision of Redmond, Wash., includes the smallportable Integrated Photonics Module (IPM) only a couple of centimeterssquare that is mounted on a HMD device. The IPM uses integratedelectronics to control a laser and bi-axial MEMS scanner to project animage through optics onto and including eyeglasses a user is wearing.Furthermore, U.S. Patent 2005/0083248, by Biocca, Frank and Rolland,Jannick et al., dated 21 Apr. 2005, entitled “Mobile face capture andimage processing system and method” disclose a camera system that looksinward to capture a user's face and not outward such that a continuouspanoramic view of the remaining surrounding scene can be recorded andinteracted with, which is critical for 2-way teleconferencing and forestablishing neural correlates of consciousness with surroundingenvironment. A further limitation of Biocca is that the cameras facinginward block the user's peripheral FOV.

Flexible electronic displays of a type integrated in the presentinvention are of a type shown in U.S Patent Application Publication2010/0045705, dated 25 Feb. 2010, Vertegaal et al., entitled“Interaction Techniques For Flexible Displays” that incorporate what isreferred to as “e-paper” in the display industry; and display screensand associated computerized image processing systems to drive flexiblethin, light-weight, either of soft or semi-rigid material, energysaving, and irregular shaped and curved LED display systems of a typeintegrated into the present invention are manufactured by BeijingBrilliant Technology Co, LTD, China, under the trade name “flexibledisplay”. It is known that non-see-through and see-through LED and OLEDsystems are manufactured. See-through LED and OLED are frequently usedin augmented reality HMD applications. Systems referenced in thisparagraph are of a type that may be integrated, retrofitted, and in somecases improved upon to realize the present invention [40].

Providing electrical power to the smartphone, portable brain activitysensing system, surround video logging system, correlation system, andsub-components are an enabling technology to the operation of thepresent invention. A conventional battery charger may be operated torecharge the battery carried by the user, typically in the smartphone.Landline transfer of energy, especially for recharging of portablesystems is well known to those skilled in the art and may be used insome embodiments of the system that comprises the current invention.However, while less common, wireless energy transfer or wireless powertransmission for recharging electrical devices is preferable because itfacilitates ease of use in some embodiments described in the presentinvention. Wireless energy transfer or wireless power transmission isthe process that takes place in any system where energy transfer orwireless power transmission. An induction charging system of a type thatmay be used to recharge devices external to the body of the user orimplanted in the user is of a type put forth in the Provisionalapplication by Ritchey et al.; U.S. patent Ser. No. 13/222,833, dated 31Aug. 2011 by Parker et al. and as US Patent Application Publication No2012/0053657 on 1 Mar. 2011 entitled “Implant Recharging”; and in U.S.Pat. No. 5,638,832, issued 17 Jun. 1997, by Singer et al., entitled“Programmable Subcutaneous Visible Implant”. Another method of providingelectrical power incorporated in the present invention is by kineticenergy replacement. Where electrical power is generated by movement andused to power electrical devices. Energy can also be harvested to powersmall autonomous sensors such as those developed usingMicro-electromechanical Systems (MEMS) technology. These systems areoften very small and require little power and whose applications arelimited by the reliance on battery power. Scavenging energy from ambientvibrations, wind, heat or light enables smart computers and sensors inthe present invention to function indefinitely. Energy can be stored ina capacitor, super capacitor, or battery. In small applications(wearable and implanted electronics), the power follows the followingcircuit: after being transformed (by e.g. AC/DC-to-DC/DC-inverter) andstored in an energy buffer (e.g., a battery, condenser, capacitor,etc.), the power travels through a microprocessor (fitted with optionalsensors) and then transmits out the gathered sensor data (usuallywirelessly) over a transceiver. Biomechanical energy harvesters havebeen created and are incorporated into the present invention. Onecurrent model is the biomechanical energy harvester of Max Donelan whichstraps around the knee. Devices as this allow the generation of 2.5watts of power per knee. This is enough to power some five cell phones.Incorporation of the above-mentioned electrical power and batterytechnologies is incorporated and anticipated in realizing the presentinvention.

Correlation processing of information from the portable brain activitysensing system, surround video logging system and other sensing systemsis a key part of the present invention. Post processing of sensor dataincludes noise filtering of brain activity data transmitted from thebrain activity sensor system, such as an AMR or other internal biometricor physiological sensor system. And also includes post processing ofexternal data representing the surrounding environment recorded bydevices such as panoramic video. A key part of the correlation is targetidentification and tracking which involves performing target recognitionand filtering out false targets. Computer software and firmware of atype that is incorporated into the present invention to filter data andmake correlations between brain pattern data and video is disclosed inU.S. Patent 2009/0196493, dated 6 Aug. 2009, by Widrow et al. entitledCognitive Method and Auto-Associative Neural Network Based Search Enginefor Computer and Network Located Images and Photographs. Hierarchicaltree and relational databases familiar to those in the computer industryand discipline are incorporated in the present invention to organize andretrieve information in the computer. Widrow teaches storing input data,images, or patterns, and quickly retrieving them as part of a computersystem when cognitive memory is prompted by a query pattern that isrelated to the sought stored pattern. Widrow teaches search, filteringtechniques, pre-processing of sensor data, post processing of sensordata, comparator operations done on data, storage of data, and keyingtechniques incorporated into the present invention. Widrow also teachesthat the computer may be part of a computer or information appliance andthat the system may be remotely connected to the global information grid(GIG)/Internet and the processes distributed. U.S. Patent Application20070124292 A1, by Kirshenbaum et al., dated 31 May 2007, entitled“Autobiographical and Other Data Collection System” teaches astereoscopic video logging system with recall. However, neither Widrownor Kirshenbaum teach a portable device for brain pattern correlationwith video logging and memory enhancement as does the present invention.And neither Widrow nor Kirshenbaum teach spherical recording with braincorrelation. Compact computer processing systems, including the latest3G, 4G, and 5G communication telecommunication systems and follow-ondevices like smartphone phones (i.e. Apple iPhone 4S and 5, Samsung Epic4G; Blackberry 4G smartphones's; chips, PCB's, DSP's FPGA's; Quantum 3DInc., San Jose, Calif., powerful compact portable computer processingand Imaging generator modules (i.e. IDX 7000, ExpeditionDI, and Thermite4110); Mini & Small PC's by Stealth Computer Inc.; the Pixel Edge Center3770 HTPC with Dual Core i7 or dual chip Xeon processors; U.S. Pat. No.7,646,367, dated Jan. 9, 2006, by Hajime Kimura entitled Semiconductordevice, display device and electronic apparatus; and associatedtelecommunications systems and methods disclosed in U.S. Pat. No.7,720,488, by Kamilo Feher, dated Jun. 21, 2007, entitled “RFID wireless2G, 3G, 4G, 5G Internet systems including Wi-Fi, Wi-Max, and OFDM” andthe like compatible and of a type incorporated into the presentinvention.

Dynamic user/host command and control of the present invention throughinteractive machine assist systems is a major feature of the aboveinvention. Interactive computer machine assist and learning systems areincorporated in the present invention to assist the host in command andcontrol of the logging and memory system. Once neural correlates areidentified using technologies specifically described in the precedingparagraph the information is referenced by artificial intelligent (AI)and AI like systems to form an enduring cognitive assistant for the useror another client in the present invention. An AI computer hardware andsoftware of a type that may be integrated with the present invention isthe Cognitive Agent that Learns and Organizes (CALO) [31, 32], developedby SRI between 2003 and 2008. CALO is a PC based cognitive softwaresystem that can reason, learn from experience, be told what to do,explain what they are doing, reflect on their experience, and respondrobustly to a client's specific commands or based on a client's repeatedactions when using the CALO system. The SIRI system is a softwareapplication on the I-Phone 4S and 5, a portable electronic device,manufactured by Apple Corporation Inc, CA. The SIRI application is apersonal assistant that learns (PAL) application that is run on theI-Phone 4S and 5. The SIRI system includes a speech recognition andspeech synthesis application that may be integrated with the smartphoneof the present invention to interact with on-board and off-systemdevices and software applications that comprise the entire system of thecurrent invention. It is an object of the present invention to integrateAI and AI-like CALO and SIRI software, Widrow's 2009/0196493 art, andKirshenbaum's logging and database software and hardware into a singleintegrated computer architecture to achieve the objectives of thepresent invention.

Microprocessor speed and memory capacity have increased along a numberof dimensions which enable the present invention. Computers get twice aspowerful relative to price every eighteen months, or in other words,increase by about an order of magnitude every five years. Additionally,decreases in size and volume of mobile computing and communicationdevices continue to make them even more portable. Bandwidth is alsoincreasing dramatically. Therefore, new uses for such powerful machines,programs, and bandwidth may be developed, as evidenced by the presentinvention. Particularly, as computing speed and memory capacity drop inprice, personal use systems become more powerful and more available.Personal communication systems, like smartphones with video cellcapability, may be in part or in whole in the present invention toprocess, display, transmit and receive data in accordance with thepresent invention. One valuable use for powerful computing processes ismultimedia, surveillance, and personal data collection. There is knownin the art individual devices which already employ microprocessors andapplication specific integrated circuits for recording specific types ofdata; e.g., video (with sound track capability) and video cameras forrecording the local surroundings (including day-date imprints), pen-sizedigital dictation devices for sound recording, space satellite connectedglobal positioning systems (GPS) for providing instantaneous position,movement tracking, date and time information, smartphone downloadablenote taking and other computing activities, biofeedback devices, e.g.,portable cardio-vascular monitors, for medical patients and sportsenthusiast, and the like. Additionally, remotely located servers may beincorporated into the present invention to receive and transmit data toand from users of the data logging and communication system comprisingthe present invention.

An additional feature of the command and control portion of the presentinvention, typically conducted by the user operating a host computer, isan integral part of the present invention. In the present invention theU.S. Patent Application 2009113298, by Edward Jung et al., dated 30 Apr.2009, entitled “Method of selecting a second content based on a user'sreaction to a first content” provides a method of a type compatible withand incorporated into the present invention. Accordingly, data sensed orrecorded by the logging and video enhancement system of the presentinvention may be operated upon in response to other data sensed orrecorded to include at least one a person's gaze, attention, gaze dwelltime, facial movements, eye movements, pupil dilation, physiologicalparameters (heart rate, respiration rate, etc.), stance,sub-vocalization (and other non-word audio), P-300 response, brainwaves, brain patterns, or other detectable aspects. In anotherembodiment, data indicative of a response may include data indicative ofat least one of a user's physiological, behavioral, emotional,voluntary, or involuntary response sensed by the system of the presentinvention.

User activation and authentication is sometimes desired when practicingthe present invention because inadvertent input might cause confusion ina host being's brain or malfunctioning in a host and remote servermachines processing. Surreptitious activation by a hostile being ormachine, either locally or remotely, could introduce unwanted input andcontrol of the host being or machine. Thus, at least standard intrusiondetection and information security systems and methods are incorporatedinto the present invention (i.e. firewalls and virus protectionsoftware). Preferably, the present system incorporates an identificationand an authentication system for activating and deactivating the systemdue to the critical nature to the user which access the presentinvention allows. It is an object to integrate and combine both standardand new novel identification (ID) and authentication systems into thepresent invention.

In some instances it may be preferable to locate at least someprocessing and database storage of the present invention at a remotelocation. This may be preferable in order to reduce weight and becauseof limited space considerations. Additionally, locating processing at aremote location may be important for safety and security reasons.

Size, location, unobtrusiveness, concealment, and support of componentsborne by the user, whether external or internal to the body of the user,are important considerations that may be addressed by aspects of thepresent invention. These requirements vary and dictate the variousembodiments of this invention. Traditional support assemblies includesecuring components onto the clothing of the user. A backpack 133 or abelt-pack are conventional examples. Distribution of some components inthe present invention is a technique used to decrease the weight andvolume of the present invention.

Improved and novel systems and methods of positioning and securingdevices to or in the host user are an important contribution andobjective of certain embodiments the present invention. These systemsand methods of dividing up and securing the components overcome many ofthe limitations mentioned above with HMD's. Alternatives include usinginvasive and/or noninvasive techniques. The present invention includesvarious systems and methods that lesson or disguise the visual impact ofpeople wearing data logging and memory enhancement systems. Pat. No.4,809,690, dated 7 Mar. 1989, by Jean-Francois Bouyssi et al., entitled“Protective skull cap 46 for the skull” is compatible and of a type thatmay be integrated into the present invention. Additionally, data derivedfrom the present invention may be transmitted for presentation by aprogrammable subcutaneous visual implant as described in U.S. Pat. No.5,638,832 by Singer in order to hide or communicate with others in thesurrounding environment in a non-verbal manner compatible with thepresent invention. Concealing implants by the use of a hair-piece, wig,fall, synthetic skin, prosthetics, optical film, skin colored and tattoosleeves, sticky material, material coverings that blend into and withthe exterior body and extremities and is an objective of the presentinvention. For instance, skull caps may be used to hide or concealcomponents of the present invention that are mounted in and on the headof the user according to the present invention. It is a furtherobjective is to integrate a covering a covering that conceals the cameraoptics comprised of a one-way film used in the optical industry oncontact lenses and eye glasses. These concealment devices are well knownto those in the medical, optical, and cosmetic industry. However, theuse of these concealment devices as described in the present inventionis not known in prior art.

In the present invention miniaturization allows sensor, input,processing, storage, and display devices to be positioned on theexterior of the user by means of conventional double sided adhesivebased techniques commonly used in the medical industry to mount heartand brain monitoring sensors to a patient. Body piercings known topeople in the body art industry are used in the present invention tosupport components of the present invention. Specifically, industrial,snug, forward helix, conch, and lobe piercings of the skin may supportcomponents. In medicine, fistula are unnatural connections or passagewaybetween two organs or areas that do not connect naturally. While,fistula may be surgically created for therapeutic reasons, in thepresent invention fistula are created to facilitate passageways forcomponents that facilitate and form the present invention. Fistula usedin the present invention include: blind—with only one end open;complete—with both external and internal openings; and incomplete—afistula with an external skin opening, which does not connect to anyinternal organ. While most fistula are in the form of a tube, some canalso have multiple branches, various shapes and sizes. In medicine, acannula is a tube that can be inserted in the body, often for thedelivery or removal of fluid. Cannula may be inserted by puncturing ofthe skin. Alternatively, cannula may be placed into the skull bydrilling or cutting a portion of the skull away and replacing it with anappropriate material or device. In the present invention fistula andcannula are used to house, support, connect, and conceal components ofthe present invention.

Subdermal and transdermal implants are known in the body modificationindustry and medical profession and adapted to the present invention tohold components of the invention in place. Subdermal implants are theimplantation of an object that resides entirely below the dermis,including (i.e. horn implants for body art: a pacemaker placed beneaththe skin for medical purposes; or a magnet implant beneath the skin toassist a user in mounting or picking up devices above the skin.) Incontrast, transdermal implants are placed under the skin, but alsoprotrude out of it. Binding and healing of the skin around and overimplants and piercings is an import part and objective of the presentinvention. Aftercare of implants is known in the body modificationindustry and medical profession and is also a part of the presentinvention. (Ref. Shannon Larratt (Mar. 18, 2002). ModCon: The SecretWorld Of Extreme Body Modification. BME_books. ISBN 0973008008) [12];(Ref. Various Medical Atlas's of Plastic Surgery, otolaryngology-headand neck (ENT) Surgery, and Neuro Surgery) [41, 42, 43].

Surgical methods used to implant components in the present invention aredescribed in various surgical atlas known to those in the medical field.Making holes in the skin and skull of living animals and insuring theirsurvival is done routinely in the medical and veterinary profession. Forinstance, a paper by Laflin and Gnad, DVM, entitled “Rumen Cannulation:Procedure and Use of a Cannulated Bovine” in 2008 by Kansas StateUniversity [13] and an article by Hodges and Simpson, DVM, in 2005entitled “Bovine Surgery for Fistulation of the Rumen and CannulaPlacement”[14] describe surgical techniques for making large holesbetween the outer skin and stomach of cattle. These techniquesdemonstrate surgical methods and the survivability of animals when largecannula and fistula are placed in animals. In the present invention,these techniques are used to make passageways for communication betweenimplanted electronic components using cannula and fistula into and onthe body of users, consistent with the present invention.

It is known in medicine that specialty implants are used in plasticsurgery to achieve aesthetic surgery. Common implants include chin,calf, pectoral, nasal, carving, and cheek bone implants. Additionally,it is known in medicine that implants are used in the body art industryto create bumps as body art. A manufacturer of such implants is SpectrumDesigns Medical, of Carpentaria, Calif. These implants may be filed withsilicone, foam, or Teflon® are typically placed just beneath the skin.In the present system implants are filled with electronic components.The components may be connected to the interior and exterior of the bodyvia fistula and cannula. Furthermore, Craig Sanders et al. demonstratein an article entitled “Force Requirements for Artificial Muscle toCreate and Eyelid Blink With Eyelid Sling” dated 19 Jan. 2010, in theARCH Facial Plastic Surg/Vol 12, No 1, January/February 2010 [15] and inan article entitled “Artificial muscles restore ability to blink, saveeyesight”, by U.C. Davis Health System, dated 11 Feb. 2010 [16]describes an implanted artificial muscle system to restore a person'seyelid blinks. The eyelid blinking system demonstrates the surgicalimplantation techniques and method of small electrical processing,battery, servos, and planted wiring beneath the skin surgical of a typeused in and enabling certain aspects of the present invention.

With respect to implants, it is known by neurosurgeons in the medicalprofession that artificial plastic skull plates may replace the skull;ref. “Applications of Rapid Prototyping in Cranio-Maxilofacial SurgeryProcedures, Igor Drstvensek et al., International Journal of Biology andbiomedical Engineering, Issue 1, Volume 2, 2008. [17] And it is known inthe electronics industry that plastic is the base material on which manyprinted circuit boards are built. Printed circuit boards aretraditionally flat, however, curved printed circuit boards have recentlybeen produced. It is an objective of the present invention toincorporate PCB technology into irregular and cranial skull plateimplants to facilitate some embodiments of the present invention.Development of curved printed circuit boards of a type that enable andare compatible with the present invention include those developed at theCenter for Rapid Product Development, Creative Research EngineeringInstitute, Auckland University of Technology, New Zealand in 2009 undertheir Curved Layer Rapid Prototyping, Conductive 3D Printing, and RapidPrototyping and Design Methodology Programs. It is therefore anobjective of the present invention to enable implantation of speciallydesigned curved and irregularly shaped printed circuit boards as asubstitute for removed sections of the skull to enable the presentinvention.

Additionally, it is an objective to use optical concealment and cloakingsystems and methods in the present invention to conceal worn devices andimplants mounted over, on top of, into, and under the skin. Systems andmethods for cloaking integrated into and compatible with the presentinvention include those described in: U.S. Patent 2002/0090131, byAlden, dated 11 Jul. 2002, entitled “Multi-perspective backgroundsimulation cloaking process and apparatus”; U.S. Patent ApplicationPublication 2002/0117605, by Alden et al., dated 29 Aug. 2002, entitled“Three-Dimensional Receiving and Displaying Process and Apparatus withMilitary Application”.

It is an object to input data and information derived by the presentinvention into a simulation system Hosts simulations of a typeconsistent with the present invention include U.S. Pat. No. 5,495,576,by Ritchey, dated 27 Feb. 1996 entitled “Panoramic image based virtualreality/telepresence audio-visual system and method”. Other enablingsimulation technology of a type compatible with and that may beintegrated into the present invention includes U.S. Patent Application2004/0032649 by Kondo et al., dated 19 Feb. 2004, entitled “Method andApparatus for Taking an image, method and apparatus for processing andimage, and program and storage medium”; U.S. Patent Application2004/0247173, by Frank Nielson et al., dated 9 Dec. 2004, entitled“Non-flat image processing apparatus, in-processing method, recordingmedium, and computer program”; U.S. Patent Application 20100030578, bySiddique et al., dated 4 Feb. 2010, entitled “System and Method forcollaborative shopping, business, and entertainment; U.S. PatentApplication 20100045670, by O'Brien et al., dated 25 Feb. 2010, entitled“Systems and Methods for Rendering Three-Dimensional Objects”; U.S.Patent Application 20090237564, by Kikinis et al., dated 24 Sep. 2009,entitled “Interactive Immersive Virtual Reality and Simulation”; U.S.Patent Application 201000115579 by Jerry Schlabach, dated 21 Jan. 2010,entitled “Cognitive Amplification for Contextural Game-TheoreticAnalysis of Courses of Action Addressing Physical Engagements”; U.S.Patent Application 2005/0083248 A1, by Frank Biocca, Jannick P. Rolandet al., dated 21 Apr. 2005, entitled “Mobile Face Capture and ImageProcessing System and Method”; U.S. Patent Application 20040104935, byWilliamson et al., dated 20040104935, entitled “Virtual realityimmersion system”; and U.S. Patent Application 2005/0128286.

Host computer servers for storing and retrieving data and informationderived by the present inventions data logging system and other socialnetwork and search engine systems operated by a user via a wirelesstelecommunication system of a type consistent with the present inventioninclude those in U.S. Patent Application 20070182812, specifically FIGS.47-51, and those above mentioned in U.S. Patent Application 20070124292A1, by Kirshenbaum et al. and in U.S. Patent Application 2009/0196493 byWidrow et al. For instance, Google Earth™ and video chat liketechnologies and graphics may be adapted as a platform for geospatialreferencing and video teleconferencing in which users of the presentinvention interact with one another. It is an objective of the presentinvention to describe a social telecommunication network that allowsusers to interactively share their thoughts and a view of themselves andtheir surrounding environments using the present invention.Telecommunications systems that are integrated with the Internet of atype that may be incorporated into the present invention to accomplishvideo communications within the scope of the present invention aredescribed in Patent Application Publication 2007/0182812 A1 dated Aug.9, 2007 by Ritchey entitled Panoramic Image-based VirtualReality/Telepresence Audio-Visual System and Method, and areincorporated by reference.

Robotic and cybertronic systems of a type that may be populated withdata derived by a data logging system of a type compatible with thepresent invention include those discussed at the: Proceedings of the18th Joint International Conference on Artificial Intelligence, Aug.9-15, 2003, Acapulco, Mexico in the article “Non-Invasive Brain-ActuatedControl of a Mobile Robot”, by Jose del R. Millan et al. [18]; theongoing NASA Robonaut 2 Program; in the scientific paper ABrain-Actuated Wheelchair: Asynchronous and Non-Invasive Brain-ComputerInterfaces for Continuous Control of Robots by F. Gal'an et al. from theIDIAP Research Institute, Martigny, Switzerland, dated 2007 [19]; U.S.Patent Application 20040104702 by Nakadai, Kazuhiro et al., dated Jun.3, 2004, entitled Robot audiovisual system; U.S. Patent Application20040236467, by Sano, Shigeo, entitled Remote control device of bipedalmobile robot, dated Nov. 25, 2004; and United States Patent Application20060241808 by Nakadai, Kazuhiro et al., dated Oct. 26, 2006, entitledRobotics Visual and Auditory System. It is known by those skilled in theart that robotic devices may be remotely piloted or operateautonomously. It is also known that robots can be programmed toreplicate characteristics of a being by translating information derivedfrom data logged about a given being and converting that data intocomputer code based on those characteristics of the living beingconsistent with some embodiments is the present invention.

Video logging and memory enhancement devices that form the presentinvention, carried on and in a being, can add additional weight.Exoskeletal systems compatible with and of a type that may beincorporated to support the additional weight of the system disclosed inthe present invention includes U.S. Patent Application Publication2997/0123997, by Herr et al., dated 31 May 2007, entitled “Exoskeletonsfor running and walking”. Passive and active exoskeletal systems knownto those skilled in the art may be incorporated into the presentinvention. An exoskeleton like that disclosed in U.S. 2003/0223844, bySchile et al., dated 4 Dec. 2003 entitled “Exoskeleton for the HumanParticular for Space Applications” which may be used for remotelycontrol of robots may be integrated into the present invention.Astronaut suites, scuba gear, other life support garb and equipment,protective garments, backpacks, helmets and so forth may be supported.Garb integrated with and supported by a user in the present inventionmay incorporate various displays, microphones, cameras, communicationdevices like cell phones, body armor, power sources, or computers andassociated devices. In one embodiment of the data logging and memoryenhancement system of the present invention, the helmet design andbackpack are supported by an exoskeletal system in order reduce theweight on the being carrying the portion of the invention born by abeing. Alternatively, the helmet design can be supported by theweightlessness of outer space or by underwater buoyancy compensationapparatus in some situations. Still alternatively, an opaque helmetdesign embodiment that captures imagery from camera systems and displaysthe imagery on the interior and exterior of the helmet is disclosed inthe present invention. Recently developed thin form flat, curved,flexible, opaque and see-through display devices known in the industryare integrated into the novel helmet design enabled various embodimentof the present invention.

Direct sensing and stimulation of existing brain cells to drive the datalogging and memory enhancement system is an objective of the presentinvention. Direct sensing and stimulation system and methods of a typecompatible and incorporated into the present invention includes: U.S.Patent 2008/0097496, 24 Apr. 2008, by Chang et al., entitled “System andMethod for Securing an Implantable Interface to a Mammal”; U.S. PatentApplication Publication 2009/0105605, dated 23 Apr. 2009, by MarcioAbreu, entitled “Apparatus and Method for Measuring BiologicalParameters”; U.S. Patent Application Publication US 2009/0163982 and2009/0306741, by Christopher deCharms, dated 25 Jun. 2009 and 10 Dec.2009, entitled “Applications of the Stimulation of Neural Tissue UsingLight”; U.S. patent application Publication, by Hogle et al., dated 10Dec. 2009, entitled Systems and Methods for Altering Brain and BodyFunctions and For Treating Conditions and Diseases of the Same”; U.S.Patent Application 20090062825, 5 Mar. 2009, by Scott Pool et al.,entitled “Adjustable Implant and Method of Use”; U.S. Patent 20090108974by Michael Deering (cited earlier); U.S. Patent Application 20020082665,by Markus Haller et al., dated 27 Jun. 2002, entitled “System and methodof communicating between an implantable medical device and a remotecomputer system or health care professional”; U.S. Patent Application20050084513, by Liping Tang, dated 21 Apr. 2005, entitled “Nanocoatingfor improving biocompatibility of medical implants”; U.S. PatentApplication 20050209687, dated 22 Sep. 2005, by James Sitzmann et al.,entitled “Artificial vessel scaffold and artificial organs therefrom”;U.S. Patent Application 20070045902, dated 1 Mar. 2007, entitled“Analyte Sensor”; U.S. Patent 20090306741, Hogle et al., dated 10 Dec.2009, entitled Systems and Methods for Altering Brain and Body Functionsand for Treating Conditions and Diseases of the Same”; atlases andarticles on Surgical Implants; and Neurosurgical Atlases familiar tothose in the medical profession. Biological material grown in vitro orex vitro containing data and/or information derived from operating thepresent invention may be implanted in the same or a different recipient.Additionally, logged data derived according to the present invention maybe incorporated into a genetically modified organism (GMO) orgenetically engineered organism (GEO) is an organism whose geneticmaterial has been altered using genetic engineering techniques. Thesetechniques, generally known as recombinant DNA technology, use DNAmolecules from different sources, which are combined into one moleculeto create a new set of genes. This DNA is then transferred into anorganism, giving it modified or novel genes. Transgenic organisms, asubset of GMOs, are organisms which have inserted DNA that originated ina different species. In such an instance, additional and enhanced sensorsystems, embedded communication devices, disease resistance, hostileenvironment survival capabilities, and superior brain and musclestrength may be engineered into the DNA such that humans with unique andenhanced-human capabilities develop from birth with data loggedaccording to the present invention recorded by a user of previousgenerations. Still further, it is an objective of the present inventionthat a cloned being may be stimulated with historical data derived fromthe data logging system in an immersive manner such that the brain ofthe cloned being is stimulated similar to that of the original beingfrom which the data was logged.

A related objective to that described in the two preceding paragraphs isloading and monitoring of implanted stem cells with data logged and dataevoked by logged data according to the present invention. Adultneurogenesis (the creation of new brain cells in adult brains) was firstdiscovered in 1965, but only recently has it been accepted as a generalphenomenon that occurs in many species, including humans (1998). Likestem cells, progenitor cells have a capacity to differentiate into aspecific type of cell. In contrast to stem cells, however, they arealready far more specific: they are pushed to differentiate into their“target” cell. The most important difference between stem cells andprogenitor cells is that stem cells can replicate indefinitely, whereasprogenitor cells can only divide a limited number of times. Systems andmethods of a type applicable to the present invention include: Thosediscussed in the International Review of Cytology, Volume 228, 2003,Pages 1-30, by Kiminobu Sugaya, University of Illinois at Chicago,entitled “Potential Use of Stem Cells in Neuro-replacement Therapies forNeurodegenerative Diseases”[20]; in Stem Cell Research & Therapy 20101:17, by Jackson et al., entitled “Homing of stem cells to sites ofinflammatory brain injury after intracerebral and intravenousadministration: a longitudinal imaging study” [21]; U.S. PatentApplication Publication 2008/0255163, by Kiminobu Sugaya et al., dated16 Oct. 2008, entitled “Use of Modified Pyrimidine Compounds to PromoteStem Cell Migration and Proliferation”; PHYSorg.com. 31 Oct. 2007.Entitled “Stem cells can improve memory after brain injury” [22]; and inMolecules 2010, 15, 6743-6758; doi:10.3390/molecules 15106743, Yong-PingWu et al., entitled “Stem Cells for the Treatment of NeurodegenerativeDiseases” [23].

Nanobots may be also be introduced into the brain of a recipient withdata and/or information derived from operating the present invention.The data and/or information may be introduced in order to reintroducelost memory to a prior user or add a new memory to a new user. Arecipient's implanted data and/or information may be derived fromanother user. Incorporating programmable nanobots and computerelectronic interfaces with brain tissue are additional methods ofsensing brain activity and introduce information derived from queries inthe present invention into the brain is a further objective of thepresent invention. It is there for an objective of the present inventionto record and incorporated information that has been logged or derivedfrom data logged using the present invention such that it may be placedin storage and then loaded into nanobots and the nanobots targeted toreplace neurons in the brain. Additionally, nanobots may be introducedinto the brain to block neural connections to inhibit or allowinformation formulated by the video logging and memory enhancementsystem according to the present invention. Nanobot technologies of atype compatible with and integrated into the present invention includethose described in the Internet video entitled “Nanobot ReplacingNeurons 3D Animation” by info@cg4tv.com dated Jun. 6, 2011[24]. The hostcomputer or a server may be used to transmit electronic signatures thruelectrodes or light fibers into the brain of the user. The stimulantsmay represent feedback responses to information queries conducted by theuser of the present. Machine interfaces to brain tissue that are of atype compatible with and integrated into the present invention include:U.S Patent Application Publication 2003/0032946 A1, dated 13 Feb. 2003by Fisherman et al. entitled “Artificial Synapse Chip Interface forElectronic Prosthetic Retina”. It is also an object of the presentinvention to disclose sensor methods and systems according to thepresent invention that may be interfaced with audio, electro-optical,and other sensors directly with body tissues according to the Fisherman'946 patent.

The data logged by individuals may be operated upon for programmingnanobots that may be introduced into the brain to restore memory orintroduce information into the neural network of the brain.Additionally, data logged by the present invention may be incorporatedin bio-engineering human systems that carry memories forward throughencoding those memories in human DNA and RNA. U.S. Patent Publication2005/0053968, by Bharadwaj et al., dated 10 Mar. 2005, and techniquesdisclosed in the UCD, Dublin, year 2012, publication Bioinformaticsarticle entitled, “DNA Data Embedding Benchmark”, by David Haughton[25], that describes a system and method for embedding information inthe DNA string while still preserving the biological meaning of thestring; is incorporated in full as a system and method of a type whichis integrated with the present invention to encode and decode raw orcorrelated information derived from the present invention into humanDNA. The logged information could may include a test file, image file,or audio file that in which large sequences are divided into multiplesegments and placed in DNA introduced to the user human or otherorganism. It is therefore an object to provide an invention that logs abeing's life experience, such that a least some portions of the loggeddata may be codified and stored into DNA and RNA, and passed to a latergeneration, as stored information in a living organism, a cadaver, ortransfer to another living being though reproduction.

Finally, in accordance with the present invention, historical data frombrain activity sensing systems, like AMR recordings, along with otherphysiological and biometric data is read into life support systems toassist in keeping a user on life support alive. Using historicalbiometric data and information from a given user derived by the presentinvention that is consistent with the user's homeostasis, when the useris a patient, can assist in making the settings of a life supportsystem(s) compatible to the specific patient. It is conceived thathistorical logged and derived from the system 100 will be used in brain,head, body or other transplants to achieve this objective.Alternatively, robotic, prosthetic, cybertronic, and robotic systems mayalso be adapted and hooked to the life support system in order toreceive and operate on the logged data derived from system 100. Brainand head transplant methods and techniques applicable to the presentinvention are disclosed by: Browne, Malcolm W. (May 5, 1998), “Essay;From Science Fiction to Science [26]; The Whole Body Transplant” in theNew York Times; by White, Robert J. [27]; as “Head Transplants” inScientific American; and in U.S. Pat. No. 4,666,425, entitled “Devicefor perfusing an animal head”.

The above-mentioned references, and the information therein, all ofwhich are distinctly different from the current invention, areincorporated by reference as enabling the present invention.

SUMMARY

An integrated human and machine portable data logging and memoryenhancement method and system are provided for interactive input/output,storage, processing, and display. More specifically, a device for theinput of human signatures from physiological and biometric sensorsrepresenting a person's internal state of being while simultaneouslyinputting signatures representing the external environment around thesame human at a given time and place are correlated into a historicalrelational database. The device includes real-time query means of thestored historical relational database for identifying correlationsbetween current internal and external signatures of the human as thehuman moves through space and time. Body integrated panoramic sensor,processing, and display devices are provided to accomplish astatistically valid “correlation of consciousness” say for example, topyramidal cells as an internal neural representation, or “cognitive map”of place and spatial cells related within the brain to externallyderived geo-spatial information and sensory representations surroundingthe user. Methods and systems are disclosed for using the resultant datafrom the data logging system as an input into a simulation, stimulation,relational database, Internet social network and search engine,telecommunications network, or emulation system within a biological,mechanical, and bio-mechanical system.

OBJECT OF THE INVENTION

It is therefore an objective of the present invention to overcome thelimitations of the above referenced and any non-referenced prior art inthe related classes and subclasses. Scientific studies show that imagesa person recalls in his or her imagination is not always as detailed ornearly as accurate as a photographic image. It is an objective of thepresent invention to provide a higher resolution and more completerecord of a being's thoughts and their environment than the mind of atypical person remembers by providing a human portable physiological(internal body) and surrounding world (external to the body) lifelogging correlation, recall, and query system. It is therefore anobjective to allow the logged information to be operated upon by theperson recording the information, another person, or a machine. It isalso an objective to optionally provide at least one remote computer tocarry out some of the computer processing operations and memory storageof information logged and derived over a telecommunications system incommunication with said user borne system of the said invention.

It is also an objective to provide a computer memory storage devicewhich periodically achieves at least some instance of the informationcaptured or derived from said user born system that comprises thepresent invention for utilization in recipient being, machine, orcombination thereof at a later time. And it is an objective of thesystem to provide a user born system that generates a databasecomprising at least some portion of the information captured and derivedfrom the user born system, which may include a neural correlates ofconsciousness (NCC) database and Conscious Percept (CP) derived by theuser born system or a remote computer.

In the present invention, low-resolution images generated fromelectrodes that are part of a brain activity sensing system may becorrelated to subjects the user focuses upon in the surroundingenvironment that are higher resolution recorded by the surround videosystem. Then when the user recalls the subject matter at a later datethe higher resolution images logged in and correlated are retrieved fromsystem memory of the host computer to assists the user in rememberingthe subject in greater detail than he can remember in his mind. It istherefore an objective of the present invention to provide a clearermore detailed method and system to call up detailed images based uponbrain activity. For instance, images recorded in the daytime by thepanoramic camera in the present invention and overlay them over adarkened space using augmented reality (AR) may be used to help the usernavigate a space in the dark. Or alternatively, to help the userremember the audio and imagery of his or her Mom who died when the userwas a young person. Still alternatively, an Alzheimer's patient may usethe historical audio and imagery to remember or restore their memory. Orstill alternatively, to simply provide a system where a person canreview their history by operating the present invention to search for anitem they lost, such as their car keys or keys to their house.

It is an objective to provide a method which operates upon at least someportion of said data and information logged and derived by said loggingand memory enhancement system, hereby referred to as the “presentinvention”, “said invention” or “said logging system”. It is anobjective of said logging system to facilitate a method of communicationbetween humans and/or machines by using computers to translate dataderived from said logging system. It is an objective of said loggingsystem to provide a method wherein said user activates a host computer(i.e. smartphone) to turn on a brain activity sensing system, surroundsensing system, and correlation system; brain activity sensing systemand surround sensing system activated to transmit respective signaturesto correlation system; and correlation system operated to identifyneural correlates of consciousness in the form of conscious percepts. Itis an objective of said logging system to operate upon signatures from aphysiological and biometric sensor system (i.e. brain activity sensor)representing a user's internal state of being at a given time and placeare input to a correlation system; while simultaneously signaturesrepresenting the external environment presented to a person (i.e. via anaudio-visual sensor system) are also input to the correlation system;wherein the correlation system operates to receive internal and externalsignatures and determines relationships between said internal andexternal signatures to define NCCs and CPs from said signatures, whichform a historical database, which is stored as a relational database;and at time two query said historical relational database to findcorrelations between current internal and external signatures; and readin said historical information from resulting from the query into saidhost computers memory (i.e. smartphone) via at least one user inputdevice (i.e. SIRI voice response to a user query) to enhance a user'sthoughts and memory based on at least one occasion that took place at agiven historical instance at a given place and time.

It is also an objective of said invention to provide a method whereinsaid user activates the host computer (i.e. smartphone) and memorycorrelation databases such that a stimulus in the environment or thoughtin the mind of the user causes brain activity; the host computer queriessaid historical database for matches between the live brain activity andthe historical brain signature database to identify similar brainactivity patterns; the matches are presented via user input devices; theuser chooses which matches to activate and act upon; and the user'sbrain is stimulated with the matched information.

It is also an objective of said invention to provide a system and methodto provide a system wherein signatures of at least one the surroundingenvironment or the individuals brain activity are input into a computersimulation system; and an objective to provide a system wherein a userwears an input device and takes the form of an avatar to interact withinthe computer simulated environment; and an objective to provide a systemwherein at least one subject in the simulation operates upon anartificial intelligence software application; and an objective toprovide a system wherein a robot or cyborg is loaded with at dataderived from at least one user brain activity sensing system, a surroundsensing system, or correlation system; and an objective to provide asystem wherein a being or machine is loaded with data derived from atleast one user brain activity sensing system, a surround sensing system,or correlation system; and an objective to provide a system wherein andan objective to provide a system wherein a being is kept alive bymechanical life support systems by using historical data derived from atleast one user brain activity sensing system, a surround sensing system,or correlation system; and an objective to provide a system wherein abeing is implanted with growth stem cells in at least one area of memoryloss; historical data derived from the surround sensing system isintroduced to the user in the form of an immersive simulation; and saidstem cells are monitored said system to determine if similar neuralpercepts are regenerated; and an objective to provide a system whereindata derived from at least the surround sensing system or correlationsystem is replayed to at least restore a being's memory or experiencethe memory of another being; and an objective to provide a systemwherein brain cells stimulated in one being using immersive simulationderived from date using at least one the brain activity sensing system,a surround sensing system, or correlation system are implanted in asecond being.

It is an objective of the present invention to provide a system andmethod that incorporate high-resolution brain activity sensing, video,and correlation systems to facilitate the detailed reconstruction ofphysiological activity, surrounding environments for historicalpurposes, such as recreating a video log of historical moments in abeing's life. It is an object of the present invention to recordmemories and thought processes and it provides several methods forpassing memory and thought processes to beings and machines. It is alsoan objective of the present invention to enable synthesizing thoughtsformed, and memories from, a plurality of beings or machines that may beplaced together to form the collective memory for a being or machine, ora combination thereof in the form of a computerized database. Forinstance, the thoughts, thought processes, and/or memories of all NobelPrize winners may be collectively gathered and joined using the presentinvention. The present invention enables near perfect recall. It is alsoan objective to extend the mortality of humans, machines, or acombination thereof, by extending their consciousness and memory beyondtheir natural body or machine lifespan via mind, body, and machinereplication, implantation, substitution. It is an objective to makehuman survival less about reproduction and age and more aboutmaintaining and growing the information of an existing being, machine,or bio-mechanical being.

It is also an object of the present invention to provide a system andmethod for incorporating computer processing to identify theabove-mentioned brain activity, identifying brain cells with relatedneural activity, identifying and relating the focus of the user'sattention to brain cells and brain activity patterns, the formation ofnew brain cells when new brain cells are formed relative to a given timeand location, given the overall physiology of the user. An objective ofthe present invention is to not only log information, but to alsooperate on the information to provide user feedback, memory enhancement,and subsequent replication of a recipient user. It is also the objectiveof the present invention to provide a system include a person to personconscious percept translation system and method; and a person to machineor machine person to conscious percept translation module. It is anobjective to provide a user identification and authentication system.

It is an objective of the present invention to provide a system forlogging life experiences comprising a personal portable computer, abrain activity sensing system, a surround sensing system, and acorrelation system. It is a further objective that the personal portablecomputer, like a smartphone iPhone S4 or S5, that has learning andcommunication capability, with applications like SIRI, CALO, iLeader,PAL, A.I., or A.I. like functionality. It is an objective to provide auser a user artificial intelligence (A.I.) system (i.e. PAL, CALO, orSIRI), or the like, that learns and communicates with at least one auser, remote user, the smartphone, or a remote computer server. Forinstance, the iLeader system does not incorporate the elementscomprising a neural activity sensor and correlation system which is anobjective of the present invention. Additionally, the iLeader systemdoes not incorporate panoramic recording capability. An objective ofthis invention is to overcome these limitations.

A fundamental object of the present invention is also to provide asystem and method of user controls for operating the present invention.It is also an objective of the present invention be primarilyinteractive, portable, and hands-free. It is also objective that thepresent invention be compatible and optionally include voice synthesis,wake-up features, sub-vocal recognition, neural activity sensing,panoramic audio-visual sensing, stereoscopic capability, multiple ROIprocessing and readout, on chip processing, infrared sensors, targettracking, eye tracking, gaze tracking, and touch sensing and variousinteractive sensory feedback hardware with computer software/firmwareapplications for input, display, processing, and feedback. It is also anobjective of the present invention that the smartphone comprises userinteractive input and presentation system for controlling the brainactivity sensing system, a surround sensing system, and a correlationsystem; wherein the smartphone includes a software application thatcomprises an Internet search engine that communicates over atelecommunication system and network over which in information derivedby the invention may be processed and shared among users of the searchengine; and wherein the smartphone includes a software application thatis part of a social network on the Internet over which at least brainactivity information, surround sensing, or correlated data derived fromthe invention may be processed and shared among users of the socialnetwork.

It is an objective of the present invention to provide body worn devicessystems and methods that sense and present information of thesurrounding environment. It is also an objective of the presentinvention that the brain activity sensing system comprises a sensorsystem that senses and records neural activity signatures by location inthe brain at a given time to enable the identification of CP's that arethe basis for an NCC database. It is also an objective of the presentinvention that the surround sensing system comprises a substantiallyspherical field-of-regard sensing system that at least includes one of aimage, audio, touch, gesture recognition, taste recording record,processing, input, and output system; and the invention includesintegrated camera and display system made of thin flexiblee-paper/material that can be shaped around a being, machine, or acombination thereof; wherein the flexible display includes anauto-stereoscopic display; and the camera of the system have thestereoscopic image capture capability; and wherein the display andassociated components may be integrated into the visual appearance ofthe user's skin, hair, eyeglasses, body form, clothing, jewelry, orsurrounding environment. For instance it is an objective to provide asurround sensing system comprising a pierced ear-ring that includes apanoramic video capture system that works in concert with a pluralityvideo camera modules to create a composite scene of the environmentsurrounding; and an objective to track at least one of the user's eyesin order for an eye tracking and gaze system to calculate the subject auser is observing in the surrounding environment for correlation with aconscious percept derived from a brain activity sensing system. It isalso an objective to provide devices and methods to surgically implantat least some portion of the personal portable computer, a brainactivity sensing system, a surround sensing system, and a correlationsystem inside the body of the user in order to conceal and facilitateportability of the invention; and a system where at least some portionof the personal portable computer, a brain activity sensing system, asurround sensing system, and a correlation system borne by the user ismounted on or about the exterior of the body of the user; such as thesmartphone, a skull-cap, an integrated camera with display OLED thumb ornoise prophetic device, or a skin colored or tattoo sleeve whose outercovering conceals a data link or electrical power link sandwichedbetween a top and bottom layer of see-through, skin colored, or tattoolooking material. It is therefore an further objective of the presentinvention to concealing the present invention borne by the user via animplant, clothing, jewelry, earbuds, EMD, skull cap, hair-piece, wig,fall, synthetic skin, prosthetics, optical film, skin colored and tattoosleeves, sticky material, material coverings and so forth that blendinto and with the exterior body and extremities and is an objective ofthe present invention so that the user of the invention fits into thepopulation without being self-conscious and uncomfortable around thepopulation not bearing the present invention.

It is an object to provide a portable computer driven integrated imagecapture system and method that comprises at least one three-dimensionalVery Large Scale Integrated Circuit (VLSIC) with at least oneRegion-of-Interest (ROI) image sensor that receives at least one imagetransmitted through at least one unexcited transparent portion of theaddressable OLED display through the objective lens system to the lightsensitive surface of the ROI image, where the OLED is located along theouter surface of the panoramic sensor; and where the remaining activeportion of the OLED that is displaying an image blocks the remainingportion of the panoramic scene from reaching the ROI sensor. It is alsoan object to provide a portable computer driven integrated capture anddisplay system and method responsive to the external and internalcontext of a being comprising a wearable image capture system withadjacent field-of-view coverage about the user that is concealed by anactive integrated display system; said display system responsive to abiometric and physiological sensor system born by the user that monitorsat least the user's brain; and a support housing to hold said integratedcapture and display system born worn by said user. It is an object toprovide a portable computer driven integrated capture and display systemwhere the capture system includes a support armature extending from theuser's head; at least one panoramic sensor head with optics forrecording all, a portion, or portions of the continuous panoramacomprising the user's face and surrounding scene; an ear bud to allowthe user to receive an audio signal; at least one microphone forsimultaneously recording at least the user's voice or surrounding audio;electrical means for driving said image sensor; transmission means forreceiving and communicating at least some portion of said signal;control means; support means on the user's head to hold said integratedlogging system born by the user. It is an object to provide a portablecomputer driven integrated capture and display system includes a userborn display device. (i.e. HMD 135 or electronic contact lenses 137.)

It is an object of the present invention to provide a system and methodthat incorporates various spatial audio sensing and presentation systemssuch as ambisonic microphones that are spatially encoded using surroundsound encoders to output spatial audio may be played back in a user'searphones or earbuds. Ambisonic microphones may be distributed in anoutward facing manner according to the present invention. Ambisonichardware known as TetraMic Spheround with associated software of a typeapplicable to the present invention is manufactured by Core Sound ofTeaneck, N.J., USA.

It is also an objective of the present invention to provide a system andmethod that incorporates conventional voice-command systems that useconventional voice recognition and voice syntheses systems like thosefound on the iPhone 4S and 5 and the SIRI systems, and to integrate theabove voice command, recognition, and voice synthesis system intovarious head gear consistent and disclosed within the present invention.

It is an objective of the present invention to provide a system andmethod that incorporates the sub-vocalization signatures of the user asan additional sensor input system in helping determine “neuralcorrelates of consciousness” to the surrounding environment and as acommand and control device to drive the memory enhancement portion ofthe present invention.

It is an object of the present system to provide a system and methodthat integrates AI and AI-like that of the CALO and SIRI software,Widrow's 2009/0196493, and Kirshenbaum's logging and database softwareand hardware into a single integrated computer system architecture toachieve the objectives of the present invention. An object of thepresent invention is to integrate these devices as components into thepresent invention in a novel manner to accomplish the present inventionand overcome the limitations of prior art.

It is an object of the present invention to operate on the entire NCCdatabase (i.e. including the raw data, processed data, and specificallythe NCC's of the being) derived by the present invention to form a basisof a relational database which may be drawn upon by a user to performvarious functions using a smartphone or the like as described in thepresent invention. It is an objective of the present invention toinclude hardware and software that facilitates recording and translatingthe uniqueness of a subject's brain and the subjects corresponding brainactivity. And additionally, to design a universal brain translationsystem and method that facilitates communication between differentbeings, machines, or a combination thereof which use different languagesto communicate with one another. Wherein at least some portion of saidinformation logged and derived (i.e. the NCC database) from saidinvention is translated between natural and machine language tofacilitate communication between humans or machines much like a “RosettaStone” for communication between beings, machines, or a combinationthereof.

It is an object of the present invention to provide a system and methodthat integrates into a self-contained life support system like anastronaut suite, scuba gear, fireman, or combat soldier wears in ahazardous environment. An objective of the present invention is toincorporate an integrated camera and display system which can beoperated to mask the wearer and yet still communicate a chosenappearance to an onlooker.

It is an object of the present invention to provide a user system andmethod wherein at least some portion of said information logged andderived by said system is introduced to stimulate existing cells or stemcells that have been implanted into a target being. It is also anobjective of the present invention is to implant electronic devices suchas microchips and nanobots to sense, stimulate, and test existing brainactivity before, during, and after information derived from the presentinvention is introduced to a recipient user.

It is an object of the present invention to provide a system and methodof presenting at least some portion of said information logged andderived from said system to a surrogate or cloned being in order tocreate a being similar to the user from which the information wasderived.

It is an object of the present invention to provide a system and methodthat comprises downloading at least some portion of said informationlogged and derived from said system into a second system that is a robotin order to create a second system with similar attributes as the beingor machine the from which the information was derived.

It is an object to provide a system and method of introducing data andinformation derived from the sensing systems of the present inventionfor input into DNA and RNA. Besides augmented cognition applicationshighlighted in the present invention, a final concluding objective is toenable beings to transfer more than just biological information forwardduring reproduction to a user's heirs and the rest of mankind. DNA withencoded information derived from the present invention is implanted intoa fertile egg or sperm of a human, embryo, or fetes, to transfer theinformation genetically using medical procedures familiar to thoseskilled in the art. For instance, an image of a being's ancestors couldbe carried forward in the DNA of the being so that the being couldaccess the image in order to see the being they evolved from. In thismanner a human may transcend or pass on to his experience in the form ofhis memories and the lessons he or she learns throughout life. Much ofthe information that comprises the individual essence of a person'sconsciousness, including thinking process, experiences, and memory, islost because of human mortality. The present invention may be used tohelp overcome that limitation by recording, storing, and reloadinglogged data into a post predecessor specimen. In the past what a personbegins life with informationally is a body with its genetic code or arobot with whatever data it has been loaded with. And in the past what aperson ends life with informationally is a body with whatever memoriesand DNA or a robot with whatever additional new stored data has beenaccumulated. It therefore conceived in the present invention thatnanobots may be programmed with data logged into and derived from thepresent video logging and enhancement system. It is also conceived inthe present invention that data logged into and data derived from thepresent video logging and enhancement system may be coded into geneticDNA or RNA which may be passed via reproduction into offspring orimplanted into other individuals. A being's experience is formed fromthe memories and connections beings construct as beings journey throughlife. This invention allows mankind to carry forth that journey withdecreased loss of information and consciousness.

Another objective is to provide a system and method comprising a lifesupport system which operates upon physiological, sensory data, andother information derived from the present invention to assist in therecovery or regeneration of a being that is injured or has an illness.

A final objective of the present invention is to provide a system andmethod for overcoming the limitations of mankind's not being able to puttheir thinking presence into created and manmade hostile environments.Examples of manmade hostile environments are burning houses, forestfires, and radioactive environments. Examples of naturally hostileenvironments are the earth's oceans and outer space. And to introducethe NCC of a being into a recipient being, machine, or bio-mechanicalsystem in order to extend the sentience of that being well beyond thenormal consciousness of that being's natural life.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1a diagrammatically illustrates a first embodiment of a system andmethod to derive Neural Correlate of Consciousness (NCC) of a useraccording the present invention;

FIG. 1b is a diagrammatic perspective of a user worn headgear comprisinga brain activity sensor system, surround audio-visual sensor system, andhost computer required to achieve the invention in FIG. 1 a;

FIG. 1c is an exterior perspective pointing out the natural lookingappearance of the user wearing the system diagramed in FIG. 1 b;

FIG. 2a is a side sectional view of a pierced earring arrangement forcapturing exterior and peripheral video logging according to the presentinvention;

FIG. 2b is a side sectional diagram with a sensor assembly thatincorporates a sub-dermal implant to hold a stud into the body of a userin accordance with the present invention;

FIG. 2c is a side sectional diagram of a body worn sensor held in placeby a double sided adhesive pad in accordance with the present invention;

FIG. 2d is a side sectional view of a magnetic implant device andarrangement for supporting an assembly located over the outer skin ofthe user in accordance with the present invention;

FIG. 2e is a side sectional diagram of a cannular or fistular implant tohold components in place in the body of the user according to thepresent invention;

FIG. 2f is a side sectional view of a skull implant device andarrangement for supporting an assembly in accordance with the presentinvention;

FIG. 3 is a exterior perspective drawing of a pierced earring with afisheye lens and microphone according to the present invention like thatwhich may be worn on the ears and/or nose in FIG. 1;

FIG. 4 is a side sectional diagram of an alternative fisheye lensarrangement for use according to the present invention like that whichmay be worn on the ears and/or nose in FIG. 1;

FIG. 5 a is a side sectional diagram of a prosthetic integrated camerasensor which may alternatively incorporate technology like that shown inFIG. 3 or 4 like that which may be worn on the ears and/or nose in FIG.1;

FIG. 5b is a side sectional diagram of an implantable integrated cameraand display sensor which may alternatively incorporate technology likethat shown in FIG. 10 or 11 like that which may be worn on the earsand/or nose in FIG. 1;

FIG. 6 is a side sectional diagrammatic illustration of an inward facingatomic magnetometer sensor array worn as headgear by the user asillustrated in FIG. 1 according to the present invention;

FIG. 7 is a side sectional diagrammatic illustration of an alternativehead borne sensor array comprising an inward facing atomic magnetometersensor array and an outward facing array of integrated microphone, laserdesignator/rangefinder or laser-radar with video camera worn by the useras headgear according to the present invention;

FIG. 8a diagrammatically illustrates another embodiment of a system andmethod to derive Neural Correlate of Consciousness (NCC) of a useraccording the present invention;

FIG. 8b is a diagrammatic perspective of another embodiment of a userworn headgear comprising a brain activity sensor system, surroundaudio-visual sensor system, and host computer required to achieve theinvention in FIG. 8 a;

FIG. 8c is an exterior perspective pointing out the natural lookingappearance of the user wearing the system diagramed in FIG. 8 b;

FIG. 9 is a side sectional diagram of the present invention whichincludes a user worn headgear comprising a brain activity sensor systemand a surround audio-visual sensor system like that shown in FIG. 8 b;

FIG. 10a is a greatly enlarged side sectional view of an integratedcamera and display system showing the image capture phase of the imagingsystem in FIG. 8;

FIG. 10b is a greatly enlarged side sectional view of the embodimentshown in FIG. 10 a showing the image display phase of the imaging systemFIG. 8;

FIG. 10c is a greatly enlarged side sectional diagram of a see-throughembodiment of the integrated display and image sensor system accordingto the present invention in FIG. 8 over the user's eyes;

FIG. 11 is a schematic diagram illustrating an auto-stereoscopic imagecapture and display system and method comprising a micro-bead array withVLSIC processing that is alternatively combined into the optical anddisplay design of the apparatus illustrated in FIGS. 10 a-c to provideauto-stereoscopic image capture, processing, measurement, and displaycapability in accordance with the present invention;

FIG. 12 is an exterior perspective of an astronaut suite whichincorporates a video logging and memory enhancement method and systemwith integrated micro-bead array capture and display as illustrated inFIG. 11;

FIG. 13a is a sectional cutaway diagram of a head covering worn by theuser according to the present invention;

FIG. 13b is an exterior perspective of the head covering shown in FIG.13 a;

FIG. 14a is a diagrammatic side view of a host computer in a backpackcabled to headgear layout of the user born portable video logging withmemory enhancement system in accordance with the present invention;

FIG. 14b is a block diagram that names principal system componentsdescribed in FIG. 14 a;

FIG. 15 is a block diagram illustrating components and interaction of anembodiment of the present invention consistent with the functionalintent of FIGS. 1, 8, and 25 of the present invention;

FIG. 16 is a cutaway perspective of an embodiment of the presentinvention comprising an electronics module comprising a host computerwith smartphone-like functionality that is mounted or implanted on theuser of the present invention consistent with FIGS. 15 and 17;

FIG. 17 is a schematic diagram showing the communication linkages anddescribing the operative functionality between the smartphone hostcomputer, sensor, and audio-visual modules illustrated in FIGS. 15-16 ofthe present invention;

FIG. 18 is a cutaway diagrammatic perspective of the in-linecommunication interconnections between the smartphone host computer,sensor, and audio-visual modules consistent with FIGS. 15-18 of thepresent invention;

FIG. 19 is a cutaway diagrammatic perspective of the wirelesscommunication interconnections between the smartphone host computer,sensor, and audio-visual modules consistent with FIGS. 15-18 of thepresent invention;

FIG. 20a is a perspective diagram illustrating the one-handactivation/deactivate and authentication sensor module of the inventionoperated by the user pressing their skin to activate an under-the-skinsensor shown in FIGS. 18-19 and FIGS. 21a -c;

FIG. 20b is a perspective diagram illustrating a two-handedactivation/deactivate and authentication arrangement which incorporatesa plurality of sensor modules shown in FIGS. 18-19 and FIGS. 21a -c;

FIG. 21a is a perspective diagram illustrating a user implantableunder-the-skin activation/deactivate and authentication sensor modulesin FIGS. 20a -b;

FIG. 21b is a plan diagram illustrating a user implantableunder-the-skin activation/deactivate and authentication sensor modulesin FIGS. 20a -b;

FIG. 21c is a side sectional diagram illustrating a user implantableunder-the-skin activation/deactivate and authentication sensor modulesin FIGS. 20a -b;

FIG. 22a is an exterior perspective view of a person wearing a head gearwhich includes a smartphone module with presentation, processing, andinput means that connects to implanted brain activity sensor system inaccordance with the present invention and consistent with FIG. 18 a;

FIG. 22b is an cutaway exterior perspective diagram of a person wearinga head gear which includes a smartphone module with presentation,processing, and input means that connects to implanted brain activitysensor system in accordance with the present invention and consistentwith FIG. 18 a;

FIG. 23a is an exterior view of a user wearing the implantableretractable electronics display module in accordance with the presentinvention and consistent with FIGS. 18a and 23b -f;

FIG. 23b is front sectional diagram showing the location of theimplantable retractable electronics display module in the head of theuser consistent with FIGS. 18a and 23 a;

FIG. 23c is side sectional diagram showing the location of theimplantable retractable electronics display module in the user's head.In this example, a portion of the skull is removed, and the device isimplanted consistent with FIGS. 18a and 23 a;

FIG. 23d is a front sectional view showing the components that comprisethe implantable retractable electronics display module consistent withFIGS. 18a and 23 a;

FIG. 23e is a front sectional view showing the components that comprisethe implantable retractable electronics display module. The module maybe connected by electrical cable or wirelessly to an electronics moduleconsistent with FIGS. 18a and 23 a;

FIG. 23f is a diagrammatic see-through axonometric schematic with arrowsindicating the motion of the retractable near eye holographic or OLEDdisplay consistent with FIGS. 18a and 23 a;

FIG. 24a is a perspective drawing of the user's thumb unit with anintegrated camera and display system in accordance with the presentinvention;

FIG. 24b is a side sectional diagram of an inductive electrical chargingarrangement that includes a boot that slips securely onto the thumb theunit like that shown in FIG. 24 a;

FIG. 24c is a sectional diagram of a prosthetic embodiment of theintegrated camera and display integrated with a prosthetic thumb worn bythe user like that shown in FIG. 24 a;

FIG. 24d is a sectional diagram of a thumb mounted integrated camera anddisplay system with electrical power and data transmitted over a smallcable implanted under the skin of the user like that shown in FIG. 24 a;

FIG. 24e is a sectional diagram of a very small electrical power cableand/or data cable run between materials that comprises a sleeve thethumb fits into that transmits power and/or data to the thumb mountedintegrated camera and display system worn by a user like that shown inFIG. 24 a;

FIG. 24f is a perspective drawing of the hands of a user wearing theintegrated thumb display with camera with induction charging receivershown in FIGS. 24a-b with an induction electrical charging transmissionsystem integrated into the steering wheel of an automobile on which thethumbs of the user are positioned in the field-of-view of the user'seyes in order to facilitate interactive video teleconferencing;

FIG. 25 is a block diagram of the portable interactive data logging andmemory enhancement system that describes the overall concept and majorcomponents of the invention;

FIG. 26a is a block diagram that describes the two basic Steps 1 and 2of operation of the portable user interactive data logging and memoryenhancement system of the present invention;

FIG. 26b is a diagrammatic representation graphically illustrating byexample the sensing and query method described in FIG. 26a that resultsin a user memory enhancement capability according to the presentinvention;

FIG. 27 is a block diagram describing Step 1, Phases 1-3 of theoperation of the portable data logging portion of the present invention;

FIG. 28 provides a diagrammatic representation of the front view of acomposite frame of undistorted panoramic imagery taken at Time 1 at agiven location by the panoramic spherical field-of-view (FOV) surroundvideo camera system of subject matter that corresponds to neuralactivity related to a conscious percept in the brain shown in FIGS. 29a-b;

FIG. 29a is a diagrammatic representation of brain imagery representingsubject matter that may be logged into the host computer system thatcorrelates with panoramic imagery shown in FIG. 28;

FIG. 29b is a diagrammatic representation of voxel brain imageryrepresenting subject matter that may be logged into the host computersystem that correlates with panoramic imagery shown in FIG. 28;

FIG. 30 is a diagram illustrating the method of constructing a computerdatabase of neural correlation tables derived from internal and externalsensor data recorded from and about a being, machine, or bio-mechanicalsystem in the present invention by operating a computerized correlationsystem;

FIG. 31 is a diagram illustrating computer normalization of commonrelationships of brain activity sensor, CP, and NCC data derived fromtwo different beings in order to construct a translation table that forma computer database to facilitate communication between two differentbeings, machines, or bio-mechanical systems;

FIG. 32 is a block diagram describing Step 2, Phases 1-6 of theoperation of the interactive portable memory enhancement method of theinvention;

FIG. 33a is a table that illustrates a more detailed description of themajor component systems, their functions, and corresponding processesthat make up the data logging and memory enhancement system described inthe present example;

FIG. 33b is a continuation of the table in FIG. 33a that illustrates amore detailed description of the major component systems, theirfunctions, and corresponding processes that make up the data logging andmemory enhancement system described in the present example;

FIG. 34 is system diagram comprising a cloud computing arrangement forvideo logging and memory enhancement wherein a local user with aportable host computer system logs data or information and/orsimultaneously queries at least one remote host computer server on theInternet in accordance with the present invention;

FIG. 35 is diagrammatic representation of a two-way telecommunicationembodiment of the invention in which a message is transmitted between aSender and Receiver which may comprise beings, machines, orbio-mechanical systems based upon a computing arrangement of the videologging and memory enhancement system in which a sending being with afirst portable host computer (i.e. a first smartphone with portablebrain activity sensor system and surround video system), a correlation,database, and query system transmits to a receiving recipient (i.e. asecond smartphone with portable brain activity sensor system andsurround video system);

FIG. 36a is a diagrammatic plan view that illustrates the overlappingpanoramic FOV coverage of the panoramic audio-visual system comprisingthe present invention of an embodiment that facilitates stereographicimaging;

FIG. 36b is a diagrammatic view that illustrates the images recorded byfisheye cameras worn by the user in FIG. 36a that result fromoverlapping panoramic FOV coverage by the panoramic audio-visual systemthat facilitates stereographic imaging in accordance with the presentinvention;

FIG. 37a illustrates overlapping hemispherical images A, B, C, and D,recorded by a panoramic camera system with four adjacent side by sidepanoramic objective lenses facing outward about a point at 90 degreeintervals along a common plane with dashed lines in the diagramindicating dynamic interactive multi-region-of-interest (ROI) areas inthe frame that are to be sampled from panoramic image frames forprocessing and display to the user or a remote user based upon theconscious percept of a user. (Reference multi-ROI sensor image datasampled out based on the eye focus of the user correlated with brainactivity data to identify CPs that forms the NCC database of the user.);

FIG. 37b illustrates a resulting frame processed for viewing by the userin which any portion of the spherical FOV panoramic scene shown in FIG.37 a may be panned and zoomed upon by using interactive input devices;

FIG. 37c illustrates a resulting frame processed for viewing by the userin which three images are sampled out of the images shown in FIG. 36 a-band FIG. 37 a;

FIG. 38 is an illustration of a screen shot of the graphic userinterface (GUI) of a host digital communication device (i.e. asmartphone, like an (Phone 4S or 5) to be acted upon by the user forinteractive command and control of hardware and firmware comprising thepresent invention;

FIGS. 39a-39e illustrate a series of alternate graphic user interface(GUI) menus displayed on a host computer operated by a user to commandand interact over the Internet using a social media and/or search engineof the present invention;

FIG. 39a is a is a graphic representation of the GUI menu displayed onthe host computer the user operates to command the host computer of thesocial media network embodiment of the present invention in which userscan share life experiences logged by operating the internal and externalsensor systems, correlation system, and host computer system thatcomprises the present invention;

FIG. 39b is a graphic representation of the GUI menu displayed of thehost computer the user operates to command the host computer to selectand record, process, store/log, and display information derived from thebrain and surround sensor systems borne by the user of the socialnetwork;

FIG. 39c is a graphic representation of the GUI menu displayed on thehost computer the user operates to designate logged selections he shareswith others on the social network;

FIG. 39d is a graphic representation of the GUI menu displayed on thehost computer the user operates to designate selections required toconduct a live video teleconference with friends logged onto the socialnetwork;

FIG. 39e is a graphic representation of the GUI menu displayed on thehost computer the user operates to conduct a search for informationusing the memory enhancement and query capabilities of the inventionborne by the user;

FIG. 40a shows a graphic representation of a GUI menu displayed on ahost computer which participants operate to conduct a videoteleconference that incorporates information derived from participantswearing the brain activity and panoramic video sensor modules inaccordance with the present invention;

FIG. 40b is a graphic representation of the interactive immersiveteleconferencing system according to FIG. 40a in which one of the usersin the teleconference has been enlarged to fill the user's display;

FIG. 41 is a block diagram describing system and method Tasks A and/orB, and Options A, B, and/or C to which data and information derived inSteps 1 and 2 during operation of the portable data logging and memoryenhancement system may be applied;

FIG. 42 is a block diagram of an example simulation system thatincorporates inputs derived from the surround sensing system and brainactivity sensing system consistent with FIG. 41 Option A and/or B inaccordance with present invention;

FIG. 43 is an exterior perspective drawing of a robotic system loadedwith data and information derived from a previous user or recipientconsistent with FIG. 41, Option C in accordance with the presentinvention;

FIG. 44 is a block diagram disclosing the general method of using theinformation derived from the data logging system to drive the actions ofa robot consistent with FIG. 41, Option C of the present invention;

FIG. 45a-d are a series of illustrations showing the present inventionintegrated into a robot with a sensor array that includes a visualsystem that comprises a camera, a three-dimensional digitizing systemcomprising a small conventional radar, and an acoustical systemincluding a microphone used for sensing and guidance consistent withFIG. 41 and FIG. 44, Option C, of the present invention;

FIG. 46 is a block diagram that illustrates a method by which historicaldata recorded and processed by the data logging system of the presentinvention is operated upon by a computer to stimulate the physiologicalprocesses, like the brain of a recipient, to re-stimulate existing braincells, stimulate the growth of new brain cells, and/or stimulateimplanted brain cells and/or regulate the physiological state of arecipient on a life support system;

FIG. 47 is a schematic diagram that illustrates systems and methods thatdata and information logged and derived from the internal physiologicalsensor systems and external surround sensing systems processed by thepresent invention may be input into a recipient being, machine, orbio-mechanical system according to FIG. 46 to facilitate enhancement,transcendence, replacement, or substitution of at least some portion ofa recipient being, machine, or bio-mechanical system in accordance withthe present invention;

FIG. 48 is a diagram illustrating the system and method of implantingstem cells in at least one area of memory loss in a region of the brain;historical data derived from the surround sensing system or brainactivity sensing system being introduced to the user in the form of animmersive simulation; and then monitoring the stem cells introduce usinga brain activity sensor system to determine if similar neural perceptsand normal brain activity is regenerated and memory is restoredconsistent with Option D described in FIG. 46 in accordance with thepresent invention; and

FIG. 49 is a schematic diagram illustrating computerized life supportsystem and method of that operates upon databases derived from internaland external data logging system according to the present invention inorder to regulate the physiological state of a recipient on a lifesupport system consistent with Option E of FIG. 46 in accordance withthe present invention.

DETAILED DESCRIPTION

Given the above listed enabling technologies the following detaileddescription is provided to demonstrate the unique, unobvious, and novelincorporation of these technologies into a design for a portable userintegrated interactive Life-Logging and Memory Enhancement Assistant(LLMEA) system and method 100 for use by a being, machine, orcombination thereof. Art cited in the “Background of Invention” areincorporated in full as enabling art of the present invention. Artdisclosed in the “Brief Description of the Drawings” and “FIGS. 1-49”are also incorporated in full as enabling of the present invention.

Various embodiments of the present invention will now be described morefully with reference to the accompanying drawings, in which some, butnot all embodiments of the invention are shown. Indeed, this inventionmay be embodied in many different forms and should not be construed aslimited to the embodiments set forth herein. Rather, these embodimentsare provided so that this disclosure will satisfy applicable legalrequirements. When the words “may”, “can”, “might”, “optional”,“alternative”, or the like are used, they mean that the associatedfeature or description is not a necessary, critical or required aspectof the broadest disclosed inventions, even though they may be desirableor preferred in certain instances. Also, please note that within thecontext of the specification the “User” wearing the portable portion ofsystem 104 comprising the invention may be referred interchangeably as abeing, specimen, person, machine, bio-mechanical system, or recipient invarious context of the present invention.

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof In the drawings, similarsymbols typically identify similar components, unless context dictatesotherwise. The illustrative embodiments described in the detaileddescription, drawings, and claims are not meant to be limiting. Otherembodiments may be utilized, and other changes may be made, withoutdeparting from the spirit or scope of the subject matter presented here.Those having skill in the art will recognize that the state of the arthas progressed to the point where there is little distinction leftbetween hardware, software, and/or firmware implementations of aspectsof systems; the use of hardware, software, and/or firmware is generallya design choice representing cost versus efficiency tradeoffs. Thosehaving skill in the art will appreciate that there are various loggingand memory enhancement embodiments of the present invention by whichprocesses and/or systems and/or other technologies described herein canbe implemented (e.g., hardware, software, and/or firmware), and that thepreferred vehicle will vary with the context in which the processesand/or systems and/or other technologies incorporated. Those skilled inthe art will recognize that optical aspects of implementations mayemploy optically-oriented hardware, software, and or firmware solutionto manipulate an image within the invention (i.e. removal of imagedistortion). Hence, many different types of wide angle and panoramiccamera systems, sensor packages, brain activity sensor and physiologicalsensing systems, wireless communication devices, correlation systems,storage systems, force feedback, and graphic user interfaces may beincorporated without departing from the scope of the invention. Thereare several possible embodiments of the logging and memory enhancementsystem of the present invention by which the processes and/or devicesand/or other technologies described herein may be effected, none ofwhich is inherently superior to the other in that any logging and memoryenhancement system to be utilized is a choice dependent upon the contextin which the logging and memory enhancement system will be deployed andthe specific concerns (e.g. portability, flexibility, or predictability)of the implementer, any of which may vary. Additionally, it will beapparent to those skilled in the art that various components andarrangements may be exercised in part or in whole to some extent withoutdeparting from the spirit of the invention.

In some implementations described herein, logic and similarimplementations may include software or other control structuressuitable to operation. Electronic circuitry, for example, may manifestone or more paths of electrical current constructed and arranged toimplement various logic functions as described herein. Electronicswithin the invention may be in the form of an IC, LSIC, VLSIC, PCB, ormotherboard. Components of the logging and memory enhancement system maycommunicate directly (i.e. over wire or fiber optics) or via wirelesstechnologies (i.e. radio-frequency, using WIFI and Bluetooth technology)known in the art, and may be supported outside or inside the human body,machine, or a combination thereof. In some implementations, one or moremedia are configured to bear a device-detectable implementation if suchmedia hold or transmit a special-purpose device instruction set operableto perform as described herein. In some variants, for example, this maymanifest as an update or other modification of existing software orfirmware, or of gate arrays or other programmable hardware, such as byperforming a reception of or a transmission of one or more instructionsin relation to one or more operations described herein. Alternatively oradditionally, in some variants, an implementation may includespecial-purpose hardware, software, firmware components, and/orgeneral-purpose components executing or otherwise invokingspecial-purpose components. For instance, in the present inventionpersonal electronic devices (PEDs), like smartphones, are a derivationof a host computer, and are referred to interchangeably depending on thecontext of the discussion. Specifications or other implementations maybe transmitted by one or more instances of tangible transmission mediaas described herein, optionally by packet transmission or otherwise bypassing through distributed media at various times. Alternatively oradditionally, implementations may include executing a special-purposeinstruction sequence or otherwise invoking circuitry for enabling,triggering, coordinating, requesting, or otherwise causing one or moreoccurrences of any functional operations described above. In somevariants, operational or other logical descriptions herein may beexpressed directly as source code and compiled or otherwise invoked asan executable instruction sequence. In some contexts, for example, C++or other code sequences can be compiled directly or otherwiseimplemented in high-level descriptor languages (e.g., alogic-synthesizable language, a hardware description language, ahardware design simulation, and/or other such similar mode(s) ofexpression). Alternatively or additionally, some or all of the logicalexpression may be manifested as a Verilog-type hardware description orother circuitry model before physical implementation in hardware,especially for basic operations or timing-critical applications. Thoseskilled in the art will recognize how to obtain, configure, and optimizesuitable transmission or computational elements, material supplies,actuators, or other common structures in light of these teachings. In ageneral sense, those skilled in the art will recognize that the variousembodiments described herein can be implemented, individually and/orcollectively, by various types of electromechanical systems having awide range of electrical components such as hardware, software,firmware, and/or virtually any combination thereof; and a wide range ofcomponents that may impart mechanical force or motion such as rigidbodies, spring or torsional bodies, hydraulics, electro-magneticallyactuated devices, and/or virtually any combination thereof.Consequently, as used herein “electro-mechanical system” includes, butis not limited to, electrical circuitry operably coupled with atransducer (e.g., an actuator, a motor, a piezoelectric crystal, a MicroElectro Mechanical System (MEMS), etc.), electrical circuitry having atleast one discrete electrical circuit, electrical circuitry having atleast one integrated circuit, electrical circuitry having at least oneapplication specific integrated circuit, electrical circuitry forming ageneral purpose computing device configured by a computer program. Inthe embodiments host, being, user, person, recipient, subject or machinemay be used interchangeably and refers to a thing or object on or intowhich the portable interactive data logging and memory enhancementsystem is situated or connected.

While line drawings are predominantly shown in the present invention toillustrate its workings and design, it should be noted that images ofhardware, software, and firmware in the real world and the actualcomponents comprising system 100 may be substituted without changing thescope of the invention. For instance, horizontal sectional line drawingsrepresenting latitudinal cross sections of the human brain are shownthat graphically represent an fMRI, AMR scan, regions, neurons,connections in the brain. And for instance, it will be understood bythose skilled in the art that related subject matter external andinternal to the body that represents a given subject may be illustratedin the drawings as line as photos, line drawings, or numbersrepresenting the same subject to help describe the invention. It will beunderstood well known to those skilled in the art that two-dimensionalimages (i.e. spectrum image, voxel based brain image, brain networkimage, etc.) or three-dimensional perspective images (i.e. spectrumimage, voxel based brain image, brain network image, etc.) may besubstituted to represent the same subject as a line drawing withoutdeviating from the spirit of the invention. And line drawingsrepresenting subjects such as people and things can be replaced withimages and photos of the actual subject without changing the disclosureof the present invention and without changing the scope and spirit ofthe invention.

Implementations of some embodiments of the present invention requirespecial medical procedures and devices that should only be implementedby a trained physician using proper medical facilities, equipment, andcare. Additionally, international, federal, and state laws may bear onand should be adhered to when considering performing an implementingsome of the embodiments of the present invention.

Furthermore, all graphic representations used as examples herein arepurely coincidental, fictitious, and any resemblance to actual people orplaces is unintentional and incidental and solely meant to illustratethe workings of the present invention. And any prior art, names ofindividuals, companies, logos, trademarks referenced in the presentinvention are meant to be used solely for a teaching tool, and aresolely owned by their agent and not claimed in any way by the presentinventor, as they are being used solely for educational anddemonstrational purposes.

The “Detailed Description” and corresponding “Drawings” are divided intothree interrelated sections to facilitate understanding and fororganizational purposes. Sheets 1-22 (FIGS. 1-24 f) describe thehardware, firmware, and software enabling the invention. Sheets 23-35(FIGS. 25-37 c) describe the processes and methods enabling theinvention. And sheets 36-50 (FIGS. 38-49) illustrateembodiments/applications of the invention that may be derived from theLife-Logging and Memory Enhancement Assistant (LLMEA) 100 system.

FIG. 25 illustrates a schematic diagram of system 100 which comprisesthe invention. The system 100 includes a portable host computer system104 that comprises a user interface that includes an interactive commandand control module 107, internal and external sensing module monitoringand logging module 109, and a correlation module 111. Module 107processes the host being 101 (i.e. also referred as a user) commandsthat control the portable computer 104 that controls module 109 andmodule 111. Command and control module 107 is operated to specify whichdata, information, and/or media content that the system 104 acts upon.Internal and external sensor data and information is transmitted to theinternal and external sensing monitoring and logging module 109. Module109 includes user physiologic activity, periphery, and surroundingenvironment sensor units. Correlation module 111 includes a feature,quality, and/or a media content identification unit commanded by controlmodule 107. Module 109 operates to command and control the transmissionof data and information to and from the correlation module 111 alongwith other stored or incoming data which may be transmitted to system104 over a telecommunications system and network 105. User commands andlogged information are operated upon to draw relationships utilizingcorrelation module 111. The device 104 is borne by the user 101 and mayinteract over the telecommunications system and network 105 with anotheruser 102 or a remote computer server system 106.

The telecommunication network 105 may include at least one remote server106 that communicates with and shares functionality with other servers,networks, and portable computers. Portions of system 100 may becomprised of separate connected components to realize portable computer104. And virtually any kind of computing device may be used to implementthe internal and external sensor system monitoring and logging module109 and correlation module 111. The system 100 may comprise variouscomponents, to include a networked computer, a server, a collection ofservers and/or databases, a virtual machine running inside a computingdevice, a mobile computing device, a PDA, cell phone, smartphone, atablet PC, a robot, or man-machine integrated system. For example,computer processing of sensor signatures gathered by module 109 andprocessed by correlation module 111 may be accomplished on one or moreremote computer server 106 systems in the world or universe or computer104 systems in the local surrounding environment 160.

Alternatively, the portable host computer 104 system may operate in astandalone mode. In such an instance, host computer 104 includes modules107, 109, 111 and other associated computer functions that areintegrated into a single device, such as a headgear 73. Optionally, inthe standalone mode, portions of host computer 104 may be distributedabout the body of the user.

Data and information derived from system 100 may be operated upon bycomputer system 104 and 106 to perform user 101 queries for informationor for social interaction. The internal sensing portion of sensingmodule 109 includes a brain 167 activity sensing system 40 that recordsdata from which self, place, and the conscious percept (CP) and theneural correlates of consciousness (NCC) of the user 101 may be derived.The CP and NCC information is derived by commanding the correlationmodule 111 to perform operations which determine the relationship to theother internal and external sensor signatures derived by the loggingmodule 109 of system 104. Data and information to operate and derived bysystem 100 may be stored on any suitable memory storage device. Memorystorage devices for use with system 104 and 106 are well known to thosein the computer industry. Additionally, data and information derived bya being 101 wearing system 104 may be input into a recipient user 110 orarchived for later use.

For example, in operation the system 100 layout described above in FIG.25 may be implemented in a computer hardware and firmware configurationshown in FIG. 1 a-c. FIG. 1a-c is a diagrammatic perspective of a firstembodiment of the present invention 100. FIG. 1a illustrates the systemand method that the present invention 100 employs to identify the NeuralCorrelates of Consciousness (NCC) 166. In this instance, the consciouspercept (CP) 161 in the mind of the user 101 in the local surroundingenvironment 160. In this example the subject of the user's CP is a “dog”159, located in the user's field-of-view 162 in the surroundingenvironment. The user 101 brain 167 activity 165 causes neurons 164 tofire and the neural network activity is detected by the brain activitysensing system 40 of system 100 which generates electronic signatures163 that are digitized and processed by system 104. The brain activity165 is correlated with panoramic video 168 imagery and audio signatures169 also captured by system 100 that are the focus of attention of theuser 101. Correlation of the brain activity and video signatures is thenderived by performing computer operations comparing the signatures inorder to deduce the conscious percept 161 and identify the neuralcorrelates of consciousness 166 of the being 101 at a given time andplace during the life of the being.

FIG. 1b is a diagrammatic perspective of a first embodiment of thepresent invention 100. In this embodiment host computer system 104comprises a smartphone 151 and a headgear 73 assembly. The smartphone151, which in this instance operates as the host computer 104, andheadgear 73 are in a communicating relationship. The headgear 73includes at least the sensing and readout portion of the internal andexternal sensing module monitoring and logging module 109 that includesa brain activity sensing system 40. The sensing and readout portion ofthe internal and external sensing module monitoring and logging module109 also includes a video camera system 2 a-2 d. Video camera system 2a-2 d includes small cameras with microphones on the ears and nose ofthe user. Camera unit 2 a is mounted on the right ear of the user, andcamera system 2 c is mounted on the right side of the user's nose. Notvisible in the drawing is camera system 2 b mounted on the left ear ofthe user, and camera system 2 d mounted on the left side of the user'snose. Additionally, and preferably, at least one left ear and/or rightaudio amplifier 138 a and 138 b (138 b is not shown) are mounted on orin the corresponding right and left ears of the user. The amplifier 138a and 138 b may comprise an earbud with a speaker. EMDs, audioamplifiers/earbuds, and microphones are in communicating relationship tomodules 107, 109, and 111 of host computer 104 by wires, fiber optics,wireless RF transceiver system, or other conventional communicationsmethods. In the FIG. 1b the host computer 104 comprises a smartphone 151carried by the user 101. Host computer 104 includes an input means andoutput means that allow the user to interactively input commands intothe control module 107. In the present example the display means is acomponent part of a mobile presentation unit that may alternativelycomprise a smartphone display and/or a wearable display, like a HMD orEMD 137 a and 137 b. In the present embodiment, at least one right eyeEMD 137 a and/or left eye EMD 137 b is mounted on the corresponding eyesof the user. The source of electrical power to the EMD and HMD may be bybattery, solar cells, or via electrical induction means, depending onthe specific design of the HMD or EMD unit.

Headgear 73 and smartphone 151 share portions of the internal andexternal sensor monitoring and logging module 109. Brain activitysensing is accomplished by sensors in the headgear. Sensor readings areread out from the headgear and transmitted to the smartphone forprocessing and storage. In FIG. 1b sensor module 109 includes a brainactivity sensing system 40 that is comprised of at least one of thefollowing: a near-infrared imaging (fNIR) unit, functional magneticresonance imaging (fMRI) unit, magneto encephalography (MED) unit,electroencephalography (EEG) unit, and/or positron emission topography(PET) unit, and other similar type of unit that senses brain activity.The brain activity sensing unit and sensor module is configured to lookat regions, neurons, and/or activity patterns of the brain in order toderive data to enable identification of Neural Correlates ofConsciousness (NCC) and conscious percepts. Emotional association,attention association, and/or cognition associations may also be deducedfrom the analysis of the brain activity of the system 100. The brainactivity 165 sensing unit transmits electronic signatures 163 of thebrain activity to the correlation module 111 located in the smartphone151. Module 111 performs measurements on the brain signatures and thenstores information about the brain activity signatures that meetscertain predetermined thresholds and rule sets stored in the computermemory of the smartphone. The exterior looking portion of the sensorsystem includes at least one video camera system 2 that faces outward,inward, or both to record the environment surrounding the user and theperiphery of the user's body. The video camera system 2 comprises aplurality of video recording devices that record image and audiosignatures representing the surrounding environment 160. Module 111stores the video in the memory of the smartphone. Alternatively, module111 transmits video to another computer or server 106 that stores thevideo in memory which may be recalled at a later date by the user 101 oranother user 102. Recall of stored information may be part of a cloudcomputing application and service of the network 105. Video is stored innon-volatile or volatile memory depending on the application specifiedby the user. Module 111 includes a content identification section thatperforms computer operations that analyze the stored brain activityinformation and video signatures to identify neural correlates ofconsciousness (NCC) 166 that relate to and identify the consciouspercept (CP) 161. As illustrated in our present example, theLife-Logging and Memory Enhancement Assistant (LLMEA) system 100identifies a “dog’ as the conscious percept 161 that the mind of theuser is focused upon in the surrounding environment 160.

Because data logging is a memory intensive system 151 may be programmedto delete old recordings after a given period of elapsed time in orderto save memory or recover memory for newer recordings (i.e. ref.Looxcie™ wearable camera system). Additionally, smartphone 151 mayinclude an automatic reaction capability that is initiated in responseto brain activity sensed by the monitoring portion of module 109. Forinstance, the system 100 may react in a predetermined manner to performvarious operations, based upon input brain activity or video imagery,when certain NCCs are activated or CPs are observed. Portable hostcomputer system 104 correlation module 109 may include an eye-trackingunit, including an iris response unit, gaze tracking unit, touchfeedback unit, and/or voice response unit that the user may control andcommand to drive system 100.

FIG. 1c illustrates the outward appearance of a non-invasive assemblythe user 101 wears to support the headgear 73 of system 100. In thepresent example, a flesh colored skull cap with brain activity sensorsystem 40 worn by the user. The skull cap 46 covers the head in a mannerthat blends the headgear 73 with the normal appearance of the user.Alternatively, other non-invasive headgear 73 of a type compatible withthe present invention may comprise a hairpiece, hood, cap, hat, helmet,or eyeglasses.

FIGS. 2a-f are side sectional views of various alternative supportassemblies and methods that are alternatively incorporated to hold theportable data logging and memory enhancement method and system of thepresent invention to the user. Assemblies supported according to thepresent invention include sensors, such as a small camera, microphone,vibration, touch, pressure, and/or brain activity sensors, and/or amicrochip, VLSIC's and/or a computer, and the like. The example supportassemblies can also be incorporated to hold head-mounted display systemsconsistent with the present invention. Assemblies may be connected toone another by wire or wirelessly. And non-invasive support assembliesmay be mounted onto the exterior skin or gear worn by the user. Invasiveassemblies are implanted into or under the skin 157 or into or beneaththe skull 13 of the user.

FIG. 2a is a side sectional view of a body pierced support assembly 3that captures and reads out video images into system 100. System 100comprises a plurality of panoramic cameras 2 that record video. In thepresent example, the optical system has a greater than 180-degreefield-of-view (FOV), that is indicated by dashed lines that designatethe FOV. Each camera 2 a and 2 d faces outward to provide adjacent FOVcoverage with adjacent cameras 2 a through 2 d. As shown in FIG. 1b thecameras 2 c and 2 d record imagery of which includes the face of theuser. Facial imagery is processed to track the direction of focus of theeyes of the user necessary to help the system 100 determine the subjectCP in the surrounding environment the user's attention is focused upon.The cameras 2 a through 2 d include optical system that focuses an imageof a portion of the environment toward which it is directed on the imagesensor 178. The image sensor reads out a video signal to the hostcomputer 104 for processing. As shown in FIG. 2a the camera is held inplace by a post 5 that penetrates through the skin 157 of the user's earwhich is held by a back or clutch 6. In the present embodiment imageryfrom the objective lens system 140 is held in place by the housing thatcomprises the hollow post 5. Imagery is projected from the objectivelens system through the post 5 to the image sensor 178 located in theclutch 6. The clutch 6 is threaded and screws onto the post. Electricalpower for the camera and electronic communication to and from the camerais transmitted through a wired or fiber optic cable 7 that traversesfrom the clutch 6 to the body worn camera 2 that is connected to theskull cap 46. Alternatively, the earring can include a battery for powerthat is located in the clutch 6. And alternatively, the electroniccommunication by between the camera unit and host computer istransmitted using radio frequency communication, in which case thetransceiver is located in the clutch 6.

FIG. 2b is a side sectional diagram with a sensor assembly thatincorporates a sub-dermal implant assembly 8 to hold a stud or lockingbar 9 into the user's body that forms a portion of the clutch 6 of theimplant. The locking bar 9 holds a rear post b which penetrates the skinin place. Standard surgical procedures are used to place the base of theimplanted stud under the skin 157. The outward end of the post 5 bscrews into the post 5 a that holds the objective lens system 140 of thecamera 2. The objective lens faces outward from the user. A rigidhousing holds all components of the implant 8 in place. The housingholds the objective lens system 140 that focuses the image onto an imagesensor 178 of post 5 a. The end of the cable 7 that penetrates theclutch 6 of the sub-dermal implant has electrical contacts that connectto the image sensor 178 when posts 5 a and 5 b are screwed together inthe operation position. Alternatively, the sub-dermal implant 8 mayinclude a battery to power the module. And alternatively, the electroniccommunication between the camera 2 and host computer 104 may comprise aradio frequency communication system. In such an instance a transceiveris located within the housing of the sub-dermal implant 8 and a hostcomputer system 104 in a communicating relationship. Video images aretransmitted to image processing portion of the video logging and memoryenhancement and awareness system of the invention for additionalprocessing and or storage.

FIG. 2c is a side sectional view of an adhesively mounted sensorassembly 12 comprising a double-sided adhesive pad 14 adhered to animage capture device 171 with transceiver 172 for capturing exteriorvideo for logging into the system 100. Double sided adhesive stick padswith a foam center are readily available at most hardware stores.Typically, the adhesive is of a type that may be removed by pealing itoff of the user. Pads of this type are used widely in the medical fieldto mount EKG sensors on human skin 157. Alternatively, the sensor unit12 may be faced outward from user and adhered to the skin or apparelworn by the user.

FIG. 2d side sectional diagram with a magnetic implant assembly 17mounted to the body comprising a magnet 18 a implanted beneath the skin157 in the form of a sub-dermal implant. An attracting metal or magnet18 b is placed on the exterior of the user that is attached to a sensorin place on the exterior of the body of the user. Implants in thepresent invention, such as the magnetic implant arrangement 17, areimplanted using surgical techniques well known within the medicalcommunity.

FIG. 2e is a side sectional view of an implanted modular assembly 11.The modular assembly is used to house and support various head and bodyborne components of the present invention. In FIG. 2e , assembly 11 maycomprise a cannular or fistular implant housing 16. The assembly 11 islocated next to the skull 13 and beneath the skin 157 of the user. Theimplanted cannula or fistula is placed within a hermetically sealed andsterilized outer membrane of the housing 16. The cannular or fistularimplant may include an access opening through the skin. The accessopening may be used as a port to perform various maintenance functions,such as for electrically recharging a battery, removing memory chips,and the like. Alternatively, the implant can be recharged remotely usinginduction charging. The cannula and fistula are implanted using surgicaltechniques well known within the medical community.

FIG. 2f is a side sectional view of a skull implant support assembly 19.The implant housing 16 is embedded as a part of the human skull 13 lyingbeneath the skin 157 of the user. The portion of the skull the implantreplaces is surgically removed. The implant is implanted using surgicaltechniques well known in the medical profession. The implant in thisembodiment of the invention is a printed circuit board constructedwithin a hermetically sealed and sterilized outer membrane of thehousing 16. The electronics are shielded within the implant the outermembrane will typically be made of a hard plastic. The implant 19 isheld by a fastening means 20 to the adjacent portion of the skull 13.The fastening means may be clips, screws, adhesive, or other well-knowntechniques known and practiced in neurosurgery. If the implant includesa battery it may be recharged remotely by using induction chargingmethods known to the medical profession, or alternatively by a kineticor wired charging system.

FIG. 3 is a perspective drawing of an embodiment of the pierced earringwith a panoramic video system 2 a, including a microphone 139, like thatworn by the user in FIG. 1b . However, in FIG. 3 the earring wirelesslytransmits a video feed to the electronics module like that detailed inFIGS. 15-17 and FIG. 19. Dashed lines with arrows indicate the directionof the on-center image path the objective lens and the audio attends toarrive at the image sensor and microphone respectively. The front post 5of the earring includes an objective lens system 140, image capturedevice 171 with transceiver 172, and microphone 139. The front post 5screws into the clutch 6 of the earring. The clutch 6 includes a battery21 which connects to and powers the electronic components in the frontpost 5. The battery may be replaced by opening a compartment on theearring clutch 6 or optionally recharged using induction charging. TheFOV of each objective lens system 140 is from greater than 180 degreesup to 220 degrees FOV. This wide FOV coverage yields a compositeadjacent spherical FOV coverage by the two objective lenses when thehemispherical images from the camera system in FIG. 2b are stitchedtogether. The edges of the two hemispherical images from a plurality ofpierced earring or nose video cameras may be stitched together by thecomputer 104 to form a spherical FOV panoramic scene using computerprocessing. Images are sampled in the panoramic scene for display on theHMD or eye mounted display (EMD) of the user.

Similarly, a body pierced support assembly 3, 8, 12, or 17 shown in FIG.2a-c , with a video camera system 2 may be mounted on the user tocapture other portions of the surrounding scene. For instance, thesurrounding scene may include the periphery of the body of the user suchas his or her eyes. For instance, in FIG. 1b , nose mounted camerasystems 2 c and 2 d and prosthetic camera systems shown in FIGS. 5a and5b have a FOV coverage that images the eyes of the user, including thepupils. The image from the cameras may be sampled and processed bysystem 100 to determine the gaze and focus of the user 101. Because thescene forward of the user is also in the FOV of the nose earring thesubject in the FOV that user is gazing or focused upon is also imaged inthe same image. Preferably a Region of Interest (ROI) image sensor isincorporated into the nose camera pierced earring that is programmed tosample the eye and the focus of the eye of the user. Each eye may beimaged in the same manner, and a stereoscopic image may be displayed.Referring again to FIG. 3, preferably the camera system 2 may include avery small microphone 139 with hemispherical coverage that faces outwardfrom the side of the user's head. Audio from the two microphones mountedon the ears of the user may be recorded achieve stereophonic sound.Still alternatively microphones may be positioned about the user's headand body to record an ambisonic spatial audio file using computerprocessing. Audio may be replayed for a user by the user wearing leftand right earbud audio speakers. The earbuds may be integrated into thedesign of the pierced earring 3. Hemispherical audio is also puttogether to form spherical field-of-regard audio coverage. Optionally,electrical power and data connection from the pierced earrings in FIG. 3is via cable running to the smartphone. Alternatively, a battery forelectrical power and transceiver for data exchange with the smartphonemay be provided for wireless connectivity. Additionally, the earring maybe integrated with the skull cap worn by the user. The skull cap 46 mayinclude an integrated camera and display system like that described inFIGS. 1a-c, 8a-c , 12, 13 a-b, 18, 19, 20, or 22. The skull cap mayinclude all components of the smartphone, surround video, brainactivity, and correlation system referred in wholes as system 1. Andinclude and connect to pierced and non-pierced support assemblies of atype shown in FIGS. 2 a-f And may include the fisheye lenses in FIG. 3-5a, and in the LED system in FIG. 5 b.

FIG. 4 is a detailed side sectional diagram of another embodiment of abody pierced support assembly 3 comprising a wireless panoramic videocamera 2 that transmits images to the smartphone according to thepresent invention. The objective lens 140 of the camera faces outwardfrom the ear of the user. The objective lens system of the camera is afisheye lens. All components are held in place by a rigid opaque housing(not shown). The image sensor is located behind the objective lens toreceive the image. A hollow cylindrical post behind the objective lensis placed through the earlobe and is inserted through the earlobe intoanother fastener or screw mount to hold the front and clutch 6 part ofthe earring together and securely to the ear of the user. The clutch 6of the earring is constructed to hold a wireless transceiver 172 whichis used to transmit video signals and receive commands from hostcomputer 104. The clutch 6 portion of the earring holds additional imageand audio processing firmware that operates upon the incoming image.Image processing may include image and audio noise reduction, imagestabilization, brightness control, barrel distortion removal, imagetranslation, and ROI sampling processing electronics. The image sensor171 may comprise a VLSIC that accomplish at least some of thesefunctions. The image sensor will typically be a CCD, CID, or CMOS (APS)sensor. Distortion removal optics 24, like Fibreye™ or panamorphic lenselements, may be incorporated to assist in removal of the barreldistortion from captured image before it reaches the sensor. Stilloptionally, fisheye lenses adjacent to one another may incorporate fiberoptic image conduits to optically translate the image into a continuousscene between the exit and entrance end of the conduit such that acontinuous and correctly oriented panoramic scene for viewing results atthe sensor end of the conduit. Still optionally, electro-optical shuttermeans 25 such as beam-splitter or micro-mirror with SLM, LCD, or an LEDshutter 179 assembly is incorporated along the optical path to block orallow images to be projected onto the image sensor as described inadditional detail in the “Related Applications” by the present inventor.Electronic circuitry 26 extends to the shutter control means from theimage sensor processing portion of the camera 171 and transceiver 172 inorder to transmit command and control signal to the shutter 179. Forexample, as illustrated in FIGS. 4 and 5 a-b, the display and integratedcamera system may operate as a shutter 179 wherein the pixel or pixelsin the user's line-of-sight 141, ROI, and CP 159 are operated to betransparent in order to allow for image sensor 171 to capture the CP159. Still alternatively, barrel distortion of the image may be removedby pre-processing of the raw image by firmware that is incorporated intothe image sensor chip, or later the electronics module or at the remoteserver. The entire image projected to the sensor from the objective lenssystem may be imaged on the sensor and relayed to the electronicsmodule. Or alternatively, pre-processing may include ROI sampling andreadout of a portion or portions of the entire image to the electronicsmodule. The clutch 6 of the earring post includes an image sensor withprocessing to facilitate readout of the video signal. The post alsoincludes a small battery that provides electrical power electricalcomponents of the earring. The unit also includes electrical power inthe form of a battery unit. The earring may be designed to have areplaceable or rechargeable battery. If the battery is rechargeable itmay be designed to be able to be recharged by an induction rechargingmethod, such as a induction charging pad (i.e. pierced-earring camera,skull cap, thumb display/camera, nose display/camera, etc.). The same istrue with charging any removable electronic devices described in thepresent invention. Video audio and imagery signals are transmitted fromthe earring to a transceiver in the electronics module of the videologging and memory enhancement system of the invention for additionalprocessing and or storage. The pierced earring camera may include amicrophone for audio pickup like that shown in the perspective shown ofan example earring shown in FIG. 3. In this manner a very low profileand commonly worn piece of body art or jewelry facilitates panoramicvideo recording for the video logging and enhancement system accordingto the present invention.

FIGS. 5a and 5b illustrate a prosthetic and implanted video camera anddisplay system consistent and enabling the present invention. The cameraand display optics supported by the user may be concealed by atransparent or semi-transparent covering material 30 tinted material, amicro-screen means, or LED and OLED display means. Such coveringmaterial is at most frequently made of rubber, plastic, or glassmaterial commonly used in the manufacturing of sunglasses, contactlenses, and electronic display screens. The prosthetic shown in FIG. andthe implant shown in FIG. 5a be concealed by any of the methods justmentioned.

FIG. 5a illustrates an integrated prosthetic camera and display assemblywith transmitter and battery 21 for the nose 25 of a user similar tothat shown in FIG. 4. The prosthetic may be introduced to repair aninjury to the nose. Or alternatively, the flesh on the lower side of thenostril is removed surgically and replaced by the prosthetic noseassembly 28. Flesh colored mascara with adhesive 29 is then used to fillin and blend the edges of the prosthetic with the nose of the user.Alternatively, skin may be grown or grafted around the display and theassembly permanently implanted in the nose of the user. Alternatively,the pierced nose assembly 3 may be attached by a clutch 6 as in FIG. 3.And optionally a microphone of less than 2 mm embedded in the display atthe edge of the display, in the display, or even inside the nose on theclutch 6 to capture the user speaking or audio signatures near the user.

FIG. 5b illustrates an integrated camera and display prosthetic patchassembly 32 for the nose of a user. The assembly 32 may comprise anarray strip of electro-optical material 46 with a stick-on backing thatcan be adhered to the nose of the user. The location of where thesticky-backed material adheres to the exterior skin of the nose isindicated by a bold solid line. The material incorporates a camera anddisplay like that shown in FIGS. 10a and 10b . The electro-opticalmaterial may be semi-flexible and contoured to the noise such that it isinconspicuous and blends in with the natural appearance of the user. Theskin color of the user may be displayed when the camera is not beingoperated to image the surrounding environment. The surroundingenvironment may include the scene surrounding the user and/or hisperiphery, to include image tracking his eyes to discern where he islooking and what CP he is focused upon. A transmitter and batteryassembly may be designed into the assembly 32 or connected to the hostcomputer 104 by circuitry, cable, fiber optics, or a transmitter inorder to receive and send data and to charge the electronic componentsof the assembly 32. Flesh colored mascara with adhesive 29 is then usedto fill in and blend the edges of the patch assembly with the nose ofthe user. Alternatively, skin may be grown or grafted around the displayof the patch to permanently implant the patch assembly on the nose ofthe user.

FIGS. 6, 7, and 9 are sectional diagrams that illustrate the brainactivity sensor system 40 that is a component of the headgear 73 of thesubject invention 100. FIG. 6 illustrates a user inward facing brainactivity sensing system 40 integrated into a skull cap 46 but no useroutward facing video sensors or display array. FIG. 7 illustrates a userinward facing brain activity sensor system 40 integrated into a skullcap 46 that includes outward facing image, audio, and other sensors butno display array. FIG. 9 illustrates a user inward facing brain activitysystem 40 integrated into a skull cap 46 that includes outward facingintegrated image sensing and display array. The array shown in FIG. 9may optionally include other integrated sensors, such as microphones139. FIG. 9 incorporates the outward facing video camera sensor and adisplay shown in FIGS. 10a-c , with an optional design embodiment shownin FIG. 11. In our present example, the headgear 73 with brain activitysensing system 40 comprises a portable Atomic Magnetometer Resonance(also referred to herein as AMR or Atomic Magnetometer Sensor ArrayMagnetic Resonance) 37 system with one or more arrays of atomicmagnetometer sensors units that detect the relaxation and excitation ofthe magnetic field induced or not induced, respectively andcorrespondingly. In the present invention one or more arrays of atomicmagnetometers directly detect relaxation of a magnetic field inducedwith subatomic precession within a target specimen. In this instance,the atomic magnetometer's sensors units are arranged in a conventionalhead worn device or helmet wherein the capacity sensors may be used ineither a scalar or a vector mode. The AMR may be used to image andprovide signal readout on anatomical and non-anatomical structures. Inthe present example, the AMR is used to record the user's brain activityas a wearable, portable array, with low power consumption, incorporatingwafer-level fabrication, with rapid signal processing, decreased needfor development of strong magnetic fields, and lower cost allowing wideravailability. Multiplexing may be utilized to periodically turn on andoff sensors to allow temporal dissipation of magnetic field effects. Inthe case of atomic magnetometers, the speed of multiplexing can belimited by the relaxation time of the gas in the detection chamber. Thisrelaxation time is typically on the order of microseconds, and is afunction of gas composition, pressure, and temperature. Therefore, thereis sufficient temporal resolution for applications such as functionalimaging. Additionally, shielding may or may not be interposed betweenspecific sensors or sensor pairs to direct magnetic field lines awayfrom adjacent sensors. As a benefit, magnetic shielding 36 (e.g.,creating a window of measurability) may augment the directionsensitivity of a given sensor or sensors. Finally, signal processing maybe utilized to focus in on or to remove known frequencies related tooperation of sensors from measurements. It should be understood, inlight of this disclosure, that many other configurations using theseconcepts are possible. Signal processing algorithms can be utilized toallow localization and deconvolution of distal signals within a targetby subtracting more proximal signals. Alternatively (or in addition),signal processing algorithms can be used to subtract environmentalnoise. Deconvolution may have the effect of reconstructing athree-dimensional map of the locations and intensities of the signalsgenerated. Because of the relatively small size of the sensors, arelatively high sensor density within a particular array of sensors maybe utilized. For example, the sensors may be placed less than 3 mm fromthe subject's scalp 34 in a closely packed array. Altering the directionof the pump or probe laser may additionally allow increased informationat the sensor for the purpose of source localization. Additionally,magnetic shielding may be interposed between the detecting magnetometerand the user specimen to constrain field detection. Shielding may insome cases comprise a disk of mu-metal or other shielding material;other configurations are possible. In some cases, shielding may berotated to alter directional sensitivity at a given sensor. Variousother dynamic shielding strategies may also be used. Various atomicmagnetometers with different detection profiles are available and thespecific strategy utilized may depend on magnetometer characteristics.Stacking and grouping of arrays of sensors or arrays of sensor clustersmay be utilized to progressively screen signal from noise and to accountfor spatially uniform sources of noise, or other externally inducedmagnetic fields. Since atomic magnetometers or similar sensors developmagnetic fields in the course of normal operation (typically related tothe direction of light propagation along the sensor), the direction oflight propagation among sensors may be alternated, or a random patternof orientation may be utilized to minimize large scale field effects. Insome cases, additional magnetic shielding (such as mu-metal shielding oractive shielding) may be placed around a sensor or a cluster of sensors,for the purpose of further mitigating inter-sensor interference, and/orin order to provide a further screen for environmental noise. Sincesensor-related magnetic fields typically have a particular magnitude andoccur at a particular frequency, signal analysis techniques may beutilized to remove the influence of inter-sensor interference from theinformation derived from the sensors. While imaging can be performedusing a pre-pulse and detection field, other additional features may beused to improve image quality. For example, Louis-Serge Bouchard, andVasiliki Demas of Berkeley Labs (Patent Pending, University ofCalifornia/Berkley, Patent ID pending) recently disclosed utilization ofpairs of rotating fields through a sample to overcome image distortionsthat typically occur when applying conventional NMR detection and MRimaging methods at low fields.

Now referring to FIGS. 1, 6, 7, 8, 9, and 14 the headgear 73communicates to the host computer via cable or wireless connection. Thehost computer may be of a conventional portable design which isfrequently implemented in portable laptops, personal digital assistants,cell phones, and the like. The host computer includes hardware andsoftware. Components are connected by a system bus and electrical busand include, but are not limited to, input/output jacks, a portablepower system with a battery, interactive input devices, video card, harddrive for storing data, random access memory for storing volatile data,central processing systems, cooling fans, telecommunications system, andthe like. Additionally, the host computer includes either software(written programs or procedures or rules and associated documentationpertaining to the operation of a computer system and that are stored inread/write memory) and/or firmware (coded instructions that are storedpermanently in read-only memory). A computer system and software of atype compatible and incorporated in the present invention is thatdisclosed in U.S. Patent 2009/0196493, dated 6 Aug. 2009, by Widrow etal. entitled Cognitive Method and Auto-Associative Neural Network BasedSearch Engine for Computer and Network Located Images and Photographs;Cognitive Agent that Learns and Organizes (CALO) [31,32] Software, andU.S. Patent Application 20070124292 A1, by Kirshenbaum et al., dated 31May 2007, entitled Autobiographical and Other Data Collection System,and IL, is a system compatible with and integrated by reference as artincorporated into the present invention is the Ultra-Vis, Leader, systemdeveloped by ARA, subsidiaries MWD, Vertek, and KAD, and other companiesto include Lockheed Martin and Microvision Incorporated teaches astereoscopic video logging system with querying [31,32]. Thus, the hostcomputer includes an operating system (OS), atomic magnetometer system,dynamic image and brain pattern activity translator and comparator,head-mounted display system (including head and eye-tracking andoptionally global positioning system), voice recognition system (andoptionally sub-vocalization system), panoramic video system, optionaltelecommunication system, and memory enhancement and personal assistantthat learns software and firmware. While preferable to use a singlecomputer language for efficiency, it will be obvious to those skilled inthe electronics and computer science that a computer program thatconverts a program from one language to another to link software writtenin a different language and machines written to run on differentsoftware together is common and may be incorporated to enable thecurrent invention if necessary. In this manner the above referencedsoftware may be linked together to form a single system in the presentinvention. This translation software may be implemented at the assemblylanguage level as a low-level programming language for computers,microprocessors, microcontrollers, and other integrated circuits; and/oras a utility program called an assembler used to translate assemblylanguage statements into the target computer's machine code.

FIGS. 6, 7, and 9 provides an enlarged side sectional views of thatillustrates the basic components and design of the headgear 73 accordingto the present invention of system 100. The focus of the AMR system willtypically and primarily be on the subject the user is focusing on in theenvironment at a given time. But it also includes other surrounding AMRbrain activity neural signatures that comprise the surroundingenvironment which stimulate place, grid, spatial view cells in thehippocampal area that provide visual cues, spatial navigation, andepisodic memories of particular locations that could be a generalmechanism responsible for the storage and recall of information about aparticular set of events which occur together at the same time.Components of the AMR portion of the head-mounted device in the figureinclude is situated on the user's head, scalp, skull, and/or brain. Inthe present invention the brain is referred to as one the areas of theinternal environment which the system 100 monitors. Insulation 38 and athree-dimensional vector, scalar, and gradient detection array includingwiring for supplying power are additional components that comprise theAMR system. Input and output signals along with electrical power areprovided through wiring and circuitry embedded in the head covering.Magnetic shielding, such as metal shielding and a noise reduction array35 is also provided in the head covering. The head covering alsoincludes an outer shell or covering of cloth, latex, rubber, plastic,Kevlar, or other suitable material. The AMR's sensors may be arranged sothat magnetic shielding is positioned between a first array of sensorsand another array of sensors. An additional layer of sensors, with eachsensor comprising one or more atomic magnetometers, is grouped outsideof a shielded region to allow for noise reduction. One or more arrays ofsensors in vector, scalar, and/or gradient mode may be utilized,depending on the application. Accordingly, the first sensor array may beutilized for signal detection, and the second sensor array may beutilized to assess the level of noise present in the signals measured bythe first sensor array. More particularly, the signals measured by thefirst sensor array may include both magnetic fields from a target areawithin the patient's body (e.g., the patient's brain) and noise.However, because the second sensor array may be shielded from magneticfield's emanating from the target area, the second sensor may measuresubstantially only the noise adjacent the first magnetometer.Accordingly, the magnetic fields from the target area may be determinedby subtracting the noise (as measured by the second array) from thesignals measured by the first sensor array. As mentioned earlier in thisapplication, an enabling technology of type compatible with the presentinvention are portable Atomic Magnetometer Sensor Array MagneticResonance (AMR) Imaging Systems and Methods devices of a type like thosedescribed in U.S. Patent 2009/0149736, dated 11 Jun. 2009 by Skidmore etal. and U.S. Patent 2010/0090697, dated 15 Apr. 2010 by Savukov havebeen disclosed that are of a type compatible and enabling of the presentinvention. John Kitching, a physicist at the National Institute ofStandards and Technology in Boulder, Colo. has developed a tiny (grainof rice size) atomic magnetic sensors of a type compatible for use inthe present invention.

Integrated with the AMR system in FIGS. 1, 6, 7, 8, 9, and 14 is apanoramic video system. The panoramic video camera system records apanoramic scene which includes the subject of visual focus in thesurrounding environment that surrounds the user. But it also includesother surrounding panoramic imagery that comprises the surroundingenvironment which stimulate place, grid, spatial view cells in thehippocampal area that provide visual cues, spatial navigation, andepisodic memories of particular locations that could be a generalmechanism responsible for the storage and recall of information about aparticular set of events which occur together at the same time. Aplurality of objective lenses are embedded in the outer covering of thehead worn device to record adjacent or overlapping imagery. The innerspace of the head-mounted assembly accommodates the AMR components andback side of the lenses and image sensor, including cabling andcircuitry, electrical power bus, control, and video input and outputsignals cabling and/or circuitry. Additionally, there is an innerspacing and lining material between the component layers of thehead-mounted assembly. It will be apparent to one skilled in the artthat various novel arrangements may be constructed according to thepresent invention to accommodate the head-mounted assembly to the user'shead, scalp, skull, and brain.

As depicted in FIGS. 1, 6, 7, 8, 9, and 14 the headgear 73 worn by theuser also includes panoramic audio recording system. The system may beseparate from the imaging system or integrated with of the imagingsystem, as is the case with most camcorder systems which record a videosignal that includes both audio and imagery. The panoramic recordingsystem may be part of simple audio system or a complex ambisonicmulti-directional audio recording system. The focus of the audio systemis primarily and typically on the subject the user is focusing on in theenvironment. But audio coverage also includes other surrounding audiosignatures that comprise the surrounding environment which stimulateplace, grid, spatial view cells in the hippocampal area that providevisual cues, spatial navigation, and episodic memories of particularlocations that could be a general mechanism responsible for the storageand recall of information about a particular set of events which occurtogether at the same time. The outer shell supports the embeddedmicrophones which includes several microphones for capturing audio ofthe subject of focus and the surrounding visual environment. The innerspace of the HMD is designed to accommodate the AMR components and backside of the microphone, including cabling and circuitry, supplyingpower, control, and video input and output signals. The inner lining ofthe head-mounted system that housing the audio sensors and associatedaudio components protects the user's scalp, skull, and brain on whichthe audio components may be situated. The audio system on the head worndevice also records voice signatures of the user. But, alternatively oradditionally, the audio system includes a throat microphone to recordthe user's voice. The voice signatures may be run through a voicerecognition system and decoded into verbal language that is translatedby the correlation system to help identify neural correlates ofconsciousness and multi-variant correlations with other sensor signatureoutput (i.e. brain activity and imagery) that is logged into thedatabase of computer.

Also as shown in FIG. 7 additional sensors that are integrated into theheadgear 73 may include a laser rangefinder/target designator andtracking system with image and pattern recognition. The output from thelaser rangefinder/target designator and tracking system with image andpattern recognition applications software or firmware is operated uponby the host computer that assists in identifying the subject or activitythe rangefinder is focused upon. Once a subject or activity is decodedthen it is correlated with other sensor information to determine thestrength of the relationship with other sensor data to see if the samedefinition is derived. If the relationship is above a set threshold,then the correlation is stored and acted upon per the predetermined ruleset established by the user.

Still further, sub-vocalization sensor unit 203, as depicted in FIG. 14,may be integrated into the headgear 73 or may be separately located onor in the user's body and feed into the host computer. In such aninstance, electrodes record an Electroencephalograph (EEG) signaturethat is processed by computer into words. The output from thesub-vocalization system with sub-vocal to word recognition applicationssoftware or firmware is operated upon by the host computer that assistsin identifying the subject or activity the sub-vocal signature isfocused upon. Once a subject or activity is decoded, then it iscorrelated with other sensor information to determine the strength ofthe relationship with other sensor data to see if the same definition isderived. If the relationship is above a set threshold, then thecorrelation is stored and acted upon per the predetermined rule setestablished by the user.

FIG. 8a-c is a diagrammatic perspective of a second embodiment of thepresent invention 100 which includes a user 101 headgear 73. In thepresent example a dog 159 in the real world surrounding environment 160in which the user 101 is wearing system 100 senses, records, processes,and presents information. The system 100 acts on brain activitysignatures 163 that equate to the Conscious Percept (CP) 161representing the dog 159 in the brain 167 of the user being 101.Information related to the Conscious Percept of the dog is correlatedwith panoramic video signatures 168 of the dog 159 to derive and definethe Neural Correlates of Consciousness (NCC) 166. The NCC, data, andinformation from which the NCC are derived represent a computerizedrelational database of information on how the user 101 perceives the dog159 in his or her mind. The headgear 73 comprises a brain activitysensor system 40 integrated into a skull cap 46 that includes an outwardfacing integrated image sensing and display array 156 and see-throughimage sensing and display array 47. The array shown in FIG. 8 mayoptionally include other integrated sensors, such as microphones 139.FIG. 8 incorporates the outward facing video camera sensor and displaysshown in FIGS. 10a-c , with an optional design embodiment shown in FIG.11. Array display systems like that shown in FIGS. 10a-c and 11a ,incorporated as display and integrated display and camera systems inFIGS. 8b-c , 9, 12, 13 a-b, and 45 a-d may partially or fully cover thebody, depending on the specific design required for a given application.For instance, an entire suite with integrated camera and display may bedesigned. In such an application camera and audio inputs from thesurround environment sensing system provide situational awareness to theuser. The brain activity sensor is notified of any threats in thesurrounding environment that that the brain activity sensing system areunaware of. The host computer system (i.e. a Smartphone with CALO andSIRI with voice command and synthesis user notification) notifies theuser of any threats in the surrounding environment (i.e. a lion, tiger,or bear), and then may activate protection measures on or off the user.For instance, the full body display worn by the user may be placed in acamouflage color to hide from a threat. Or as indicated in FIG. 15, aservo 181 may activate a repellant dispenser to release shark repellantto scare of the threat (i.e. a shark around a scuba diver). Displaysfacing outward may display text to other people around the user and inline of sight of the user, and HMD 135, electronic contacts EMD 137, andthe like facing inward may provide text and other graphic messages tothe user in an augmented reality manner.

FIG. 9 is a diagrammatic sectional of the headgear 73 with theintegrated brain activity sensor system 40 with an integrated displayand panoramic camera array 156 and 47 as depicted in FIG. 8a-c . Theskull cap is shaped around the ears, eyes, nose, and mouth of the userand may be folded up like a stocking cap. The head covering isconstructed of molded in latex, or other flexible material to allow theskull cap to fit adjacent and firmly against the user's head.Bi-directional arrows in the diagram emanating from the brain activitysensor system indicate that the brain activity sensor emits signals andcollects return signals to sense brain activity. Bi-directional arrows49 in the diagram emanating to and from the integrated display andpanoramic camera system indicate illumination outward of the display andinward collection of image signatures by the panoramic camera system.

FIGS. 10a-b is a greatly enlarged side sectional diagram of anintegrated flexible display and image sensor system according to thepresent invention. An addressable outward facing integrated display andpanoramic camera array 156 compatible and consistent with the presentinvention 100. The assembly comprises a digitally driven micro-mirrordevice used in the present invention for projection of a type that isoperated in the array 156 to reflect the image to the image sensor 178or alternatively to open so that an image is displayed. The device 174functions as a shutter and reflective device in array 156. A digitallydriven micro-mirror 174 device used in for projection and shuttering ofan image of a type that is used in the present invention is of a typemanufactured by Texas Instruments as a DLSP. The sensor 178 may comprisea VLSIC 180 on-chip dynamically addressable multi-window ROI imagingsensor. A sensor 178 of a CCD or CMOS type that may be incorporated inthe present invention is manufactured by JPL, Nova, Dalsa, and PhotonicVision Systems (PVS) Inc. Imaging chips manufactured by these entitiesinclude special circuitry that allows imagery recorded by an individualpixel or group of pixels within the imaging array to be read out. Onceread out the image signals may be processed on-chip, sent to aprocessing chip on the same or an adjacent printed circuit board, or anadjacent computer for processing. Examples of such ROI chips includethat found in U.S. Pat. No. 6,084,229 by Pace, and U.S. Pat. No.5,541,654 by Roberts, U.S. Pat. Pub. 2004/0095492 by Baxter, theentirety of all being incorporated by reference. Each photo diode 177,photo sensor 178, and micro-mirror 174 reflector/shutter are addressableby a computer electronic control unit/CPU that is integrated into theVSLIC 180 or optionally electronic system 104. As shown in FIG. 10a , inoperation a micro-mirror shutter 174 is closed to reflect an incomingimage transmitted through an objective lens 140 and focusing and relaylens 176 to an image capture sensor. In the example shown in FIG. 10athe eye 225 of the user 101 is being imaged when the array 156 is in thetaking mode as the user 101 views the surrounding environment 160. Oralternatively, as shown in FIG. 10b , in operation a micro-mirrorshutter 174 is open to reflect an outgoing image on the display 177through a relay optic 182 and 140 lens system to a viewer 101. In theexample in FIG. 10 b the face of the user 101 is being displayed to aviewer 295 by the portion of the array 156. It should be noted that thearray may be operated so that some portions of the array are displayingand other portions are imaging, and also that this operation may done ina user selected and/or automated manner by the user commanding the hostcomputer, which is in communication with the array. For instance, oneportion of the array may be operated to image the eye of the user, whileanother portion of the array is imaging the surrounding scene, whilestill another portion of the array is displaying an image to the user oran onlooker.

As shown in FIG. 10a , in operation a micro-mirror shutter is closed toreflect an incoming image transmitted through an objective lens and 140lens to an image capture sensor 178. In FIG. 10a light rays 48 from asubject 161 image are reflected through the micro-bead lens array 50,through open micro mirror 174 shutters, and to a photo sensor 178.Sensor 178 reads out the imagery through video drive circuitry 54 to theCPU of VLSIC 180 or host computer 104. Micro-mirror DLSP shutters areopen and closed in response to commands by the VLSIC 180 or system 104.Commands to close a mirror, and capture an image are determined by VLSIC180 or system 104. Or alternatively, as shown in FIG. 10b , in operationa micro-mirror shutter is open to reflect an outgoing image from an LEDphoto diode 177 through a relay optic 176 and an objective lens 140 ofthe micro-lens array 50 to a viewer in the surrounding environment. Thehost computer 104 reads out imagery to the display diode 177 throughdrive circuitry 49. Micro-mirror 174 DLSP shutters are open and closedin response to commands by the VLSIC or system 104 commands to reflectan image to the eye of a local user 101, onlooker 161, or a remote user102. Interactive input devices to drive the capture and display array156 include image sensor 178, target tracking, eye tracking, voice totext recognition, sub-vocal, and neural sensing systems integrated withsystem 100 consistent with the objectives of the present invention. Thelens array 50, and for that matter the array 156, may be integrated intoa rigid or flexible material. A plurality of lenses comprise the lensarray 50 in the present view. Still referring to FIG. 10a-b ,individuals skilled in the art will realize that various states ofdisplay and capture states are possible by addressing the individualphoto sensors, display diodes, and micro-mirrors as depicted in FIGS. 8a and 9. Optionally, the array 156 may be powered by an onboard batterysystem, photovoltaic methods, or remote power source located on hostcomputer system 104 in a communicating relationship to the array 156.And optionally, at least one microphone 139 is embedded into or alongthe edges of the array 50 as illustrated in FIGS. 8 and 9.

FIG. 10c is a greatly enlarged side sectional diagram illustrating auser 101 see-through sensor and display system embodiment 47 of thepresent invention 100. The illustration shows the user's view to theoutside world 160 of the integrated display and image sensor systemaccording to the present invention. An image of a dog depicts the user'sconscious percept 159. The line of sight from the eye of the userthrough the LED 177 display and through micro-mirror 174 to the outsideworld is shown by a dashed line 162. The integrated display and sensorsystem may be comprised of a flexible LED or OLED display withintegrated image sensors. Micro-mirrors 174 are opened and closed to letlight pass or not pass onto the see-through display diode(s) 177 orphoto/image sensor(s) 178 for each individual segment of the array.Relay 182 and/or focusing lenses 176 are placed in the optical path ofthe elements of the array to allow the user to look through the array tothe outside world or be reflected by a micro-mirror 174 to a sensor 177or 178 as dictated by the host computer and as dictated by the functionthe user is performing. Optionally, the user may wear relay or focusinglenses 176 placed outside the sensor system 47 in the line of sight ofthe outside world and user's eyes to accommodate the user's vision. Inthe present example the user is looking through the left side of thearray at a dog in the real world, while a right side of the arraydepicts the photo sensor on the user side of the array displaying animage to the user's eye to augment the outside reality laid over thereal world, and the right side of the array on the outward side recordsan image of the dog in the real world. Obviously, if oriented toward theuser's face the photo sensor could record the user's eyes and a videotarget tracking system can determine the direction and gaze of the userto determine that the subject the user is focused upon is a dog. Theimage of the subject is then correlated with other sensory data andbrain activity data to identify the subject as a dog and build a“Rosetta Stone” of the mind that corresponds and forms relationships toimages, text, voice recordings, etc. Finally, an entire surroundingenvironment may be dynamically or selectively operated by the usercontrol of the smartphone to record the scene surrounding the user. Theimage sensors of the micro-bead lens array 50 is made of flexiblematerial that may be faced in all directions about the user to recordthe panoramic scene which is logged by the system 100. The LED diodes177 are see-through when in a state not required for display. Stillreferring to FIG. 10c , individuals skilled in the art will realize thatvarious states of display, capture, and see-through are possible byaddressing the individual photo sensors, display diodes, andmicro-mirrors as depicted in FIG. 8 a.

FIG. 11 is a schematic diagram illustrating an auto-stereoscopic imagecapture and display technique and system 144 comprising a micro-beadarray with VLSIC processing. Sensors and optical elements depicted inFIG. 10 a-c may be implemented in FIGS. 11a-c to achieve multi-region ofinterest on-chip processing for eye-tracking and subject of interestdisplay. And the auto-stereoscopic imaging technique and systemillustrated in FIG. 11 may be combined with the optical and displaydesign of the apparatus illustrated in FIGS. 10 a-c to provideauto-stereoscopic image capture, processing, and display capability inaccordance with the present invention. CMOS multi-ROI tracking andimaging sensors may be incorporated in the design of the integratedcapture and display system shown in FIG. 10a-c based on thefunctionality and design required by the user. If required, image offsetprocessing and calculations are determined on the VLSIC CPU processor ora remote electronic device. U.S. Pat. No. 5,724,758, dated 6 May 1998,by Gulick, Jr. and U.S. Pat. No. 2,833,176, dated 21 Jul. 1953, by A. J.L. Ossoinak entitled disclose a type of optical system that iscompatible and incorporated into the present invention shown in FIG. 11for recording and displaying imagery auto-stereoscopically. The imagerecorded by each optical bead is project out from the display in thesame orientation it was recorded from whence it was recorded. The effectseen by the user is auto-stereoscopic viewing. Preferably, the systemsand methods shown in FIGS. 10a-c or FIG. 11 are incorporated into thegarb shown in FIG. 12, 13 a-b to achieve auto-stereoscopic display andviewing. Additionally and alternatively, U.S. Pat. No. 7,808,540, byCok, dated 5 Oct. 2010, entitled “Image capture and integrated displayapparatus”; U.S. Pat. No. 7,714,923 entitled “Integrated display andcapture apparatus”; and U.S. Pat. No. 7,697,053 entitled “Integrateddisplay having multiple capture devices” are of a type that may beincorporated into the present invention.

The present invention 100 may incorporate any one of a number oftraditional eye tracking or head tracking systems. Examples of eyetracking and target detection systems of a type enabling and of a typefor incorporation into the present invention have already been presentedin the above background disclosure of the present invention and are wellknown to those skilled in the art. Many navigation systems, surveillancesystems, weapon systems, and self-protection systems provide a user witha video image of a region of interest from which the user may wish todesignate an object or feature for tracking. The eye tracking systemmonitors the position of a user's eye within its socket in order todetermine the user's line of gaze. The present invention incorporates avideo tracking system borne by the user to track the eyes of the user.For instance, in FIGS. 1b, 10a, 10c, and 11b provide examples in which anoise camera or a see-through display tracks the eyes of the user. AndFIG. 8b provides an example where the eyes are tracked by the camerathat is part of the opaque or see-through integrated camera and displaysystems presented in FIG. 10c and/or FIG. 11. From either of theseeye-tracking methods data is provided to define the conscious percept ofthe user. Optionally, it should be noted that autonomous-tracking ofhazardous or items of interest in the surrounding environment may beprogrammed into the host system 100. In this manner a self: protectionor situational awareness system could be realized.

For example, FIG. 12 is a perspective of an astronaut suit whichincorporates the video logging and memory enhancement system and methodthat comprises the present invention 100. FIG. 13a is a cutawayperspective utilizing the head covering illustrated in FIGS. 10a-c and11. In FIG. 12 the display is integrated into an astronaut ExternalVehicle Activity (EVA) system worn by the user 101 of system 100. Inboth FIGS. 12 and 13 a-b the headgear 73 includes a display array 47,144, and/or 156 that forms the outer surface of the visor of the helmet.The array 47, 144, and/or 156 facing the user's face illuminates andrecords the user's face. Then the VSLIC processes the image andtransmits the image for display on the outward facing side of the arrayon the exterior of EVA helmet so that an onlooker may view the user ofthe EVA's face. An advantage of this helmet design is that it offersgreater protection to the user's head because a thicker protectivematerial may be used between the inward and outward facing arrays of theEVA helmet, yet still allows facial communication with another person inthe surrounding environment. The EVA system such as life support systemmay include propulsion, waste, food, and a breathing apparatus plusintegration of the components that comprise the present invention 100.The headgear 73 may be designed with hard or flexible material. Andheadgear 73 and integrated system 100 may also be designed toaccommodate various environmental situations and conditions, such asouter space or under water environments.

FIGS. 13a and 13b illustrate an embodiment of the present invention thatcomprises a system that acts as a head covering and poncho worn by theuser. The headgear 73 is preferably supported by an exoskeletalstructure 201. The purpose of the exoskeletal structure is to supportthe weight of the helmet that holds the life logging and enhancementsystem. FIG. 13b is a perspective drawing illustrating exterior of thehead covering 43. The exterior of the head covering may be flat or of acurved material. FIG. 13a is a cutaway perspective showing the interiorof the hemispheric shaped head covering. Typically, the base, orsupporting structure of the headgear 73 is made of a rigid compositeplastic, such as Kevlar. However, various substrate material may be usedin constructing the helmet in order to achieve less or greater strength,protection, and weight. Besides rigid material, the headgear 73 may beconstructed of a flexible material. Device drivers able to addressimaging sensor and displays of irregular shaped coverings, such ascircular patterns, are known in the display industry. The curved displayscreen preferably comprises at least an area in front of the user's facein focus graphics and imagery on the display array facing inward. Theuser's face may also be illuminated by the interior facing display.Facial actions may be sensed by monitoring brain activity or by usingimaging systems that monitor facial expressions. The array image sensormay be operated to record imagery for logging and transmission. Forinstance, facial expressions of the user may be displayed on the outwardfacing surface 43 of the external display array of the visor so that theuser may interact and communicate with other beings while the user isstill being protected inside the helmet. Besides the user's facialfeatures, the helmet may display colors that make it blend in with thesurrounding environment. The colors may blend in with the backgroundwhich make the head covering especially tailored for wild game hunting.And the entire covering of the user, including the clothing and vehiclethe user is using may covered with a similar exterior display.Microphones and image sensors embedded in the exterior of the displayhelmet or elsewhere record the images and audio of the surroundingenvironment. The recorded images and audio are processed by a computerfor input to the user. Audio speakers embedded on or into the headgear73 adjacent to the left and right ear front and back of the user 101 mayprovide directional binaural audio to the user. Image sensors may beplaced facing outward from and embedded in or on the headgear orelsewhere on the user capture video imagery that is displayed to theuser. In the present system a curved display system is embedded on orinto the interior side of the helmet. Optical systems that provideintegrated image capture and display of a type that may be incorporatedinto the present invention are those described in U.S. patent Ser. Nos.11/621,150, 12/375,805, U.S. Pat. Nos. 4,769,292, 4,928,301, 5,061,569,5,194,955, 5,639,151, 5,777,665, 6,072,496, 6,454,414, 6,771,303,6,888,562, 7,042,486, 7,474,799, 7,697,053, 7,714,923, 7,808,540, and7,808,540. For user respiration, heating, ventilation, and airconditioning a life support system may be included for the user as partof the head covering. These systems are preferably located in a suite orbackpack 133 unit worn by the user that circulates air up into thehelmet. A movable visor may optionally be constructed in the front ofthe user's face. The visor may be raised and closed manually ormechanically by the user.

Now referring to FIGS. 14a-14b that illustrate the components, layout,and interaction of the portable body borne system 100. In our presentexample, the internal portion of the head worn system includes brainactivity sensor system 40 comprising an Atomic Magnetometer Resonance(AMR) 37 system with one or more arrays of atomic magnetometer sensorsunits that detect the relaxation of the magnetic field induced. In thepresent invention one or more arrays of atomic magnetometers directlydetect relaxation of a magnetic field induced with subatomic precessionwithin a target specimen. In this instance the atomic magnetometerssensors units are arranged in a conventional head worn device or helmetwherein the capacity sensors may be used in either a scalar or a vectormode. The AMR may be used to image and provide signal readout onanatomical and non-anatomical structures. In the present example the AMRis used to record the user's brain activity as a wearable, portablearray, with low power consumption, incorporating wafer-levelfabrication, with rapid signal processing, decreased need fordevelopment of strong magnetic fields, and lower cost allowing wideravailability. Multiplexing of brain activity signals from the AMR systemmay be utilized to periodically turn on and off sensors to allowtemporal dissipation of magnetic field effects. In the case of atomicmagnetometers, the speed of multiplexing can be limited by therelaxation time of the gas in the detection chamber. This relaxationtime is typically on the order of microseconds, and is a function of gascomposition, pressure, and temperature. Therefore, there is sufficienttemporal resolution for applications such as functional imaging.Additionally, shielding may or may not be interposed between specificsensors or sensor pairs to direct magnetic field lines away fromadjacent sensors. As a benefit, magnetic shielding (e.g., creating awindow of measurability) may augment the direction sensitivity of agiven sensor or sensors. Finally, signal processing may be utilized tofocus in on or to remove known frequencies related to operation ofsensors from measurements. It should be understood, in light of thisdisclosure, that many other configurations using these concepts arepossible. Signal processing algorithms can be utilized to allowlocalization and deconvolution of distal signals within a target bysubtracting more proximal signals. Alternatively (or in addition),signal processing algorithms can be used to subtract environmentalnoise. Deconvolution may have the effect of reconstructing athree-dimensional map of the locations and intensities of the signalsgenerated. Because of the relatively small size of the sensors, arelatively high sensor density within a particular array of sensors maybe utilized. For example, the sensors may be placed less than 3 mm fromthe subject's scalp in a closely packed array. Altering the direction ofthe pump or probe laser may additionally allow increased information atthe sensor for the purpose of source localization. Additionally,magnetic shielding may be interposed between the detecting magnetometerand the user specimen to constrain field detection. Shielding may insome cases comprise a disk of mu-metal or other shielding material;other configurations are possible. In some cases, shielding may berotated to alter directional sensitivity at a given sensor. Variousother dynamic shielding strategies may also be used. Various atomicmagnetometers with different detection profiles are available and thespecific strategy utilized may depend on magnetometer characteristics.

Stacking and grouping of arrays of sensors or arrays of sensor clustersmay be utilized to progressively screen signal from noise and to accountfor spatially uniform sources of noise, or other externally inducedmagnetic fields. Since atomic magnetometers or similar sensors developmagnetic fields in the course of normal operation (typically related tothe direction of light propagation along the sensor), the direction oflight propagation among sensors may be alternated, or a random patternof orientation may be utilized to minimize large scale field effects. Insome cases, additional magnetic shielding (such as mu-metal shielding oractive shielding) may be placed around a sensor or a cluster of sensors,for the purpose of further mitigating inter-sensor interference, and/orin order to provide a further screen for environmental noise. Sincesensor-related magnetic fields typically have a particular magnitude andoccur at a particular frequency, signal analysis techniques may beutilized to remove the influence of inter-sensor interference from theinformation derived from the sensors. While imaging can be performedusing a pre-pulse and detection field, other additional features may beused to improve image quality. For example, Louis-Serge Bouchard, andVasiliki Demas of Berkeley Labs (Patent Pending, University ofCalifornia/Berkley, Patent ID pending) recently disclosed utilization ofpairs of rotating fields through a sample to overcomes image distortionsthat typically occur when applying conventional NMR detection and MRimaging methods at low fields.

Still referring to FIGS. 14a-b , the headgear 73 communicates to thehost computer 104 via cable or wireless connection that is in a backpack133. The host computer 104 may be of a conventional portable designwhich is frequently implemented in a portable laptop, personal digitalassistant, smartphone, cell phone, and the like. The host computerincludes hardware and software with an operating system (OS) andapplications required to achieve the functionality of the disclosedinvention. Components are connected by a system bus and electrical busand include, but are not limited to, input/output jacks, a portablepower system with a battery, interactive input devices, video card, harddrive for storing data, random access memory for storing volatile data,central processing systems, cooling fans, telecommunications system, andthe like. Additionally, the host computer includes either software(written programs or procedures or rules and associated documentationpertaining to the operation of a computer system and that are stored inread/write memory) and/or firmware (coded instructions that are storedpermanently in read-only memory). A computer system and software of atype compatible and incorporated in the present invention is thatdisclosed in U.S. Patent 2009/0196493, dated 6 Aug. 2009, by Widrow etal. entitled Cognitive Method and Auto-Associative Neural Network BasedSearch Engine for Computer and Network Located Images and Photographs;Cognitive Agent that Learns and Organizes (CALO) Software, and U.S.Patent Application 20070124292 A1, by Kirshenbaum et al., dated 31 May2007, entitled Autobiographical and Other Data Collection System, and ILis a system compatible with and integrated by reference as artincorporated into the present invention is the Ultra-Vis, Leader, systemdeveloped by ARA, subsidiaries MWD, Vertek, and KAD, and other companiesto include Lockheed Martin and Microvision Incorporated teaches astereoscopic video logging system with querying [31,32]. Thus the hostcomputer includes an operating system (OS), atomic magnetometer system,dynamic image and brain pattern activity translator and comparator,head-mounted display system (including head and eye-tracking andoptionally global positioning system), voice recognition system (andoptionally sub-vocalization system), panoramic video system, optionaltele-communications system, and memory enhancement and personalassistant that learns software and firmware. While preferable to use asingle computer language for efficiency, it will be obvious to thoseskilled in the electronics and computer science that a computer programthat converts a program from one language to another to link softwarewritten in a different language and machines written to run on differentsoftware together is common and may be incorporated to enable thecurrent invention if necessary. In this manner the above referencedsoftware may be linked together to form a single system in the presentinvention. This translation software may be implemented at the assemblylanguage level as a low-level programming language for computers,microprocessors, microcontrollers, and other integrated circuits; and/oras a utility program called an assembler used to translate assemblylanguage statements into the target computer's machine code.

Referring again to FIG. 14a the focus of the AMR system will typicallyand primarily on determining the CP the user is focusing upon in theenvironment at a given time. But brain activity signatures outside theCP may be also be sampled and acted upon. Brain activity neuralsignatures that stimulate place, grid, spatial view cells in thehippocampal area and that provide visual cues, spatial navigation, andepisodic memories of particular locations that may be a generalmechanism responsible for the storage and recall of information about aparticular set of events which occur together at the same time.Components of the AMR portion of the headgear 73 in the FIG. 14a-b maybe situated on or in the user's head, scalp, skull, and/or brain,respectively. In the present invention the brain is referred to as oneof the areas of the internal environment which the system 100 monitors.

Integrated with the AMR system in FIGS. 14a-b is a panoramic videosystem. Still referring to FIGS. 14a-b , the headgear 73, worn by theuser also includes panoramic audio recording system. The headgear 73comprises audio output systems such as speaker system, such as ear budaudio speakers 138, may provide audio input to the user. Many of thevideo camera 2 system current video encoding formats carry high fidelityaudio. Such audio data can be passed along with a pseudo cone pixel datastream (PCPDS) for a contact lens display, or separated out within aheadpiece. Binaural audio can be brought out via a standard miniheadphone or earbud jack, but because the system in many cases will knowthe orientation of the head (and thus the ears) within the environment,a more sophisticated multi-channel audio to binaural audio conversioncould be performed first, perhaps using individual HRTF (head relatedtransfer function) data. Feed-back microphones in the ear buds allow forcomputation of active noise suppression by the audio portion of theheadpiece. The speaker can receive input via a radio frequency signalfrom a remotely located source with audio communication capabilities. Oralternatively may be connected via wires to a unit that provides audiosignals for amplification to a small speaker in the ear. Small earphones and ear buds that fit into and onto the ear are known to those inthe art and are commonly used in the hand-free cell phone industry andsecurity industry which are of a type that is compatible with andincorporated into the present invention. U.S. Patent 20080056517 byAlgazi et al., dated 6 Mar. 2008, entitled Dynamic Binaural SoundCapture and reproduction in focused or Frontal Application that is of atype compatible with and incorporated in the present invention. Algazidiscloses a method of tacking head motion and providing directionalaudio to a headphone or earbud that may be incorporated in the presentinvention. Still referring to FIGS. 14a-b , additional sensors that areintegrated into the head worn assembly may include a laserrangefinder/target designator and tracking system with image and patternrecognition. A sub-vocalization system may be integrated into the headworn device or may be separately located on or in the user's body andfeed into the host computer.

FIG. 15 is a block diagram of another embodiment showing the interactionof the components of the present invention 100 consistent with theoverall workings and intent of the invention shown in FIGS. 1, 8, and16-25. In addition to the WAN interface module 78 operation described inFIG. 15 and FIGS. 25-40 a-b, host computer system 104 includes a LANmulti-channel transceiver 79, such as Bluetooth means, for receiving andsending a plurality of wireless data transmissions to input sensors inthe headgear 73 that drive the operation of the present invention. Inoperation host computer system 104 includes a brain activity processingmodule 81′ and correlation and query module 75 for identifying neuralcorrelates of consciousness from brain signatures; voice recognitionmodule 74 for processing speech commands or sub-vocal signatures; animage processing module 72 for eye tracking, facial imaging, featuretracking, and panoramic imaging, and optional servo 181 control forimplanted devices via interface module 71 that are centrally commandedby the central processing unit (CPU) assembly 56. Transceiver 79transmits data, typically by radio frequency data link 76, to thewireless on body servo mechanisms, sensors, and displays which have abuilt-in transceiver that may comprise distributed pieces in acommunicating relationship worn or carried by either user 101 or 102 orformed into a collective support assembly 82, such as a single headgear,worn by either user 101 and 102. The image processing module 72 operatesupon information that is transmitted to the contact lens display(s) 137located in or on the user's eyes or HMD 135. The image processing module72 operates to define the user's visual FOV and subjects of interest fortracking based on information received from the spherical FOV camerasystem 2 with ROI tracking, display, and processing. Data derived fromthe image processing module is sent to the servo control module 75 whichis in turn sent to transceiver 79. Transceiver 79 transmits the data tooptional servos 181 a-nth. Servos may be used to operate variousoptional mechanical devices born by the user 101 that are stimulated bycommands initiated by information derived from system 100. Conflictsbetween sensor commands are resolved by the central processing assembly56 or the interface module 71. The purpose of the computer processingunit (CPU) 56 is to serve as the master control system for all modulesof system 104 in headgear 73, and in communications with and overnetwork and system 105. The modules of system 104 and headgear 73communicate with the CPU 56 over a bus that transfers data betweencomponents of system 100. System 104 includes a battery module, whichsupplies electrical power to the computer modules.

Still referring to FIG. 15, the purpose of the interface module 71 isoperates to route incoming and outgoing signals between system 104 andon body user mounted mechanisms, sensor, and display systems with thetransceiver in the WAN interface module 78 of telecom module 58 andlocated in headgear 73. It will be known by those skilled in the artthat there are many tradeoffs in functionality of the invention that maybe realized in firmware or hardware. For instance, the image processingmodule may be integrated solely in firmware resident to the CPU 56 andnot include a separate processing module 72. Additionally, functionalitymay be divided up either on the wireless communication device 44 orintegrated into the wireless on body mechanisms, sensors, and displayswith transceivers 58 without departing from the spirit of the presentinvention. For instance, ROI processing can be placed on the camera 2system sensors, image processing module 72, or divided up between both.These are tradeoffs the designer of the system may make in tailoring thesystem to a particular application.

FIG. 16 is a cutaway perspective of an embodiment of the presentinvention wherein a host computer 104 electronics module 33 comprises adevice with smartphone-like functionality. FIG. 16 provides anembodiment in which system 100 that may be realized as a collectivesupport assembly 82 that comprises a single headgear 73 incorporatingmodules in smartphone 151 that includes a host computer 104 andcomponents in headgear 73 described in FIG. 15 into electronic module 33and headgear 73 embodiments disclosed in FIGS. 18-19 worn by either user101 or 102 or recipient user 110. Referring again to FIG. 16, the module33 is constructed for implanting or mounting on the body of the user 101of the present invention 100 consistent with FIGS. 2a-f , FIGS. 15, 17,and 18. FIG. 16 shows a cutaway sectional diagram of the side of theelectronics host computer module. In FIG. 18 the electronics module 33is located beneath the skin of the user shown in a see-throughperspective of the head of the user. The electronics module 33 isinserted as a fistula or cannular implant. As shown in the detailedsectional diagram in FIG. 16, the access port 210 allows exterior entryto the interior each of the electronics module located in the temporalarea of the user's head. The access port comprises a mini-USBinput-output jack that has clips in and out, such that when depressedallows the removal of USB and access to the interior components of theelectronics module. When inserted, the USB jack facilitates direct wiredcommunication of the host electronics module with other computers orsub-modules. Various jacks, like 1394 jacks, known to those skilled inthe art, may be substituted to accomplish the same functionality.Electrical power is transferred to the to a battery sub-module of thehost computer electronics module 33. The electronics sub-modules includeelectrical power and communications busses that route information andelectrical power to appropriate locations that allow the electronicsmodule and sub-modules to function typical to a conventional computer.When the access port is removed processing, communication, memory,battery sub-modules, and the like, may be accessed within theelectronics module housing. The sub-modules are inserted and removedthrough the access port when the USB input-output jack is removed. Thesub-modules are held in place by the walls of the surrounding housingand tension using an arrangement similar to that found in a Pez®dispenser or ammunition clip. Alternatively or additionally, thesub-modules may be held in place by gravity and the walls of thesurrounding housing. The electronics module is surrounded by a housingthat separates the electronics in the electronics module from adjacentparts of the body. Preferably the edges of the electronics module arerounded and medicated and placed in a hermetically sealed housing so notto irritate the skin and other body tissue. The electronics module maybe constructed of a thin rigid or flexible material and surgicallyimplanted inside the user or mounted outside the head of the user withthe flat part tangential to the head of the user. The electronics modulematerial is treated with materials and medications that allow the moduleto avoid rejection and to be accepted and integrated into the humanbody. Mounting and implantation procedures of devices comprising thepresent invention, like the electronics module and other that or placedin and on beings, are known to those in the medical and veterinarianprofession, as well as those in the skilled in body art.

FIG. 17 is a schematic block diagram of the electronics module 33 whichgenerally illustrates module functionality and connectivity consistentwith FIGS. 16, 18, and 19. Preferably, the electronics module 33 has avery small volume, which may be embodied as a Smartphone, or in evensmaller embodiment like a VLSIC. The communications module 58 that ofthe host computer 104 electronics module 33 preferably includes leastencryption firmware, Mux/Dmux, CODEC, individual sensor switching andaddress electronics sensor processing and one wireless transceiver. Theelectronics module 33 may be designed to have a standalone ornon-standalone capability. In its non-standalone mode, the electronicsmodule receives signals in and out over WAN Network 78: (i.e. RemoteServer on the Internet) (Optional: Encryption, Mux/Dmux, CODEC, WirelessTransceiver). In its standalone mode the electronics module receivessignals between to/from local Input Devices 79: (i.e. HMD, ElectronicLens Contact Display, Ear buds) (Optional: Encryption, Mux/Dmux, CODEC,Wireless Transceiver). Local input devices the electronic module 33communicates with the headgear 73 and includes but is not limited to: abrain activity sensor system 40 module like AMR, EEG, MRI Sensor(s) 81(Optional: Encryption, Mux/Dmux, CODEC, Wireless Transceiver); Sub-vocalSensor(s) 80 (Optional: Encryption, Mux/Dmux, CODEC, WirelessTransceiver); Imagery Sensor(s) 77 (Optional: Encryption, Mux/Dmux,CODEC, Wireless Transceiver); Audio Sensor(s) 138 (Optional: Encryption,Mux/Dmux, CODEC, Wireless Transceiver). The modules in FIGS. 16 and 17function as what is popularly referred to as a host computer, portableelectronic device, and personal digital assistant, to control the system100. Those skilled in the art will realize from looking at FIGS. 16 and17 that the smartphone 104, electronics module 33, and headgear 73 withsensors can be designed to be wired or wireless and the LLEMA system 100can be realized as a separated unit as depicted in FIGS. 14a and 14b ,or a single headgear 73 as depicted in FIG. 18-19.

In FIG. 18 video signals are transmitted from image sensors 2 a-d andaudio sensors 138 a-b mounted on the user over conventional signaltransmission means 206, such as wiring or fiber optic image conduits,implanted under the sub-dermal skin but above the skull of the user. Thevideo is feed to the electronics module 33 detailed in FIGS. 16 and 17.The video sensors 2 a-d have fisheye objective lenses facing outward ina sideways position such that the surrounding environment and user'sface, including the eyes of the user are imaged. Of special importanceis that the objective lenses 2 c-d each respectively have aline-of-sight view 141 c-d to the pupil on their respective sides of theface. A video imager records the location of the pupils of the user'seyes. The location of the pupils is operated upon by the host computer104 to calculate the direction of gaze and focus which in turn allowsthe calculation of the location of the subject the user is viewing. Alaser designation system 41 or LADAR system 42 may be mounted on theuser's head to mark or image targets respectively. Systems 41 and/or 42,like that in FIG. 6b , is also embedded by a sub-dermal punch orsurgically implanted into the center of the head of the user andconnected to the electronics module. Additionally, sub-vocalizationsensors 80 a-b are located on the throat of the user. The sensors areconnected to the electronics module 33 by shielded wire or fiber opticcable that is implanted beneath the sub-dermal layer of the skin. In theinstance with the sub-vocal sensors the implanted line is run from thesensors across the throat, to the neck, and forward the ears to theelectronics module 33.

The brain activity sensing system 40 includes brain activity sensors(AMR sensors) 81 a-c and 81 d-f, shown in FIG. 18 and FIG. 19, thattransmit out brain activity 165 signature 163 readings in the form ofelectronic signals to the electronics module 33. For example, in FIG. 19AMR sensors 81 a-c and 81 d-f are connected to the electronics module byshielded wire or fiber optic cable 206 et al that is implanted in thebrain 167. In the present example cluster of three sensors are spacedapproximately 10 mm apart facing toward a region of the brain called thesupplementary motor area in the upper central part of the brain about20-30 mm just below the scalp. When neurons in this region become activeit indicates the user is indicating or thinking a “yes” response. In thepresent example a cluster of three other AMR sensors are connected tothe electronics module by shielded wire or fiber optic cable that isimplanted beneath the sub-dermal layer of the skin in the upper backpart of the throat. In the present example three other sensors arespaced approximately 10 mm apart facing toward this region of the braincalled the parahippocampal gyms area in the lower central part of thebrain. When neurons in this region in this region and the supplementarymotor area become active it indicates the user is indicating or thinkinga “no” response. As mentioned earlier, “yes” or “no” responses can bepaired with menus presented to the user on head-mounted display 211 a-b(per FIG. 18) or electronic eye mounted display (EMD) 137 a-b (see FIG.1 a) to define how a user feels about all manner of subjects andactivities. These responses can then be logged and retrieved in responseto future queries. The embodiment shown FIG. 19 teaches that all devicesdepicted in FIG. 19 may be at least one connected, powered, andcommunicate to the electronics module 33 mounted on or within the uservia shielded wire or fiber optic cable implanted beneath the sub-dermallayer of the skin or in the brain 167. Implanted lines, such as 206 etal, run from the sensors to the electronics module 33. In this manner,correlations between the audio, imagery, sub-vocalization, and brainactivity is calculated into neural correlates that translate andcorrespond to subjects and activities observed externally or thought ofin the mind of the user, even without an external stimulus beingpresent.

In contrast to FIG. 18 and FIG. 19 illustrates that sensors may transmitreadout signals wirelessly to the electronics module 33 described inFIGS. 16 and 17. Wireless transmission will typically be accomplishedusing radio frequency transmission. For example, in an embodiment of theinvention, the module 33 includes a transceiver, as do the sensors orsensor systems, which include user exterior outward and user peripheralfacing imagery sensors 77, exterior inward facing sub-vocal sensors 80,internal brain activity sensors system 40 with MM sensors, and userexterior located outward and peripheral facing audio sensors 138.Electronic multiplexing and demultiplexing (Mux/Dmux) systems andcompression and decompression (CODEX) systems may be incorporated inmodule 33 and the sensor systems to assist with transfer, parsing, androuting the signals electronically as described in FIG. 15-17. In FIG.18 the module 33 and sensors each include a transceiver for sending andreceiving data to and from the electronics module. In this instance, themodule and sensors each include a transceiver for sending and receivingdata to and from the electronics module. On/Off buttons 209 a-b, whichare pressure sensitive, are implanted below the skin and connected tothe electronics module. The On/Off buttons are operated by the user tolog in and off of the system 100 as described in FIGS. 20a-b and 21a-c .In the present embodiment, an electronics module 33 also comprises asmall portable host computer 104 with many features similar to a smallsmartphone 151 that is constructed to be implanted into the user 101.The wireless embodiment of the system includes a digital switchingsystem that allows specific components to be addressed. Specifically,sensors are addressed wirelessly using the micro-circuit sensorswitching system. Electrical power for the electronics module isprovided by a battery. The battery is charged using a mini-USB or othertype of conventional charging attachment. Alternatively, the electronicsmodule and sensors are implanted beneath the skin 157 with no cannularor fistular opening to the outside of the body. In such an instance, theelectronics module and sensors batteries are charged by an electricalinduction charger. The electronics module includes a charging padlocated on the outer skin side of the module. Sensors also includeelectronics to receive electrical power for charging. An inductioncharging pad 191 may be worn continuously on the user's head, neck, orbody to facilitate recharging of the electronics module and sensors. Toreduce the weight on the user's head, a backpack 133 or belt isincorporated to hold batteries that provide current to an induction padhead charging skull cap arrangement that holds the charging pads thatsurround the user's head. The battery may be embedded in a bed pillow sothat the user can charge the battery at night while sleeping. The modulebattery provides electrical power to the electronics of the electricalmodule and the sensors. Sensors receive electrical power and data overshielded and clad wires or fiber optic conduits. All surgical implantsare sterilized and implanted according to known medical methods andtechniques to insure proper healing of the incision associated with theinvasive implant surgery.

The present invention is preferably equipped with user interactiveOn/Off and security authentication means. For example, in FIGS. 20a-b ,the location of On/Off sensor modules 209 a-b are shown located on thehead of the user 101. To activate and deactivate the system, the userpresses at least one finger on the skin where a module 209 a-b islocated to activate the under-the-skin touch sensor module 209 shown inFIGS. 21a-c . FIG. 20a is a perspective diagram illustrating theone-hand On/Off activation/deactivate shown in FIGS. 18-19 and FIGS. 21a-c.

FIG. 20b is a perspective diagram illustrating a two-handedactivation/deactivate shown in FIGS. 18-19 and FIGS. 21a-c . FIG. 21a isa perspective diagram illustrating the user implantable under-the-skinactivation/deactivate and authentication sensor modules 209 a-b in FIGS.20a-b . FIG. 21b is a plan diagram illustrating a user implantableunder-the-skin activation/deactivate and authentication sensor module209, also illustrated below the skin as sensor modules 209 a-b in FIGS.20a-b . FIG. 21c is a side sectional diagram illustrating a userimplantable under-the-skin activation/deactivate and authenticationsensor modules 209 a-b in FIGS. 20a-b . An example of how the system 100“On/Off” function operates is that the user 101 simultaneously pressessensor module 209 a-b button 213 a-b simultaneously and thinks of anauthentication code or password. The password is registered by thesub-vocalization system and brain activity sensor system of system 100.The housing 16 is preferably in a surgically-implanted, flexible, latex,hermetically-sealed and medicated housing that allows module 33 buttonsto be depressed. Button 213 is spring loaded and may be depressed by theuser to complete a circuit that sends an On/Off electrical signal to thehost computer 104 electronics module 33 to activate or deactivate thesystem 100. The activation module may be constructed and implanted in asimilar manner as the electronics module 33 described in FIG. 16. Thesignature is checked in the permissions, user ID, or password database.If the proper identification procedure is followed, and the properinformation provided to the host computer, the video logging system isturned on. The same procedure may be repeated by the operator to turnthe video logging and enhancement system off. Using this method helpsensure authorized use of the system because the procedure requires aunique action and identification code that only the user knows and mustconsciously implement. It also decreases the chance of accidentallyturning the system on or off. Still alternatively, various otherbiometric techniques and sensors can be used to activate the system 100.For instance, a heat sensor that senses the user's fingers over the skincould be substituted for the current arrangement shown to command thehost computer 104 electronic module 33 on and off.

FIG. 22a-23f are diagrams of two embodiments of a near eye displayassembly 217 consistent with the present invention. FIG. 22a is anexterior view of person wearing the near eye display assembly 217 on thetemporal sides of the head of the user 101. The display assembly 217 maycomprise one assembly or two sub-assemblies 217 a and 217 b, referred tocollectively using reference 217 a-b, that join together at area 145. Asdemonstrated in the current example, the display assembly 217 may alsoinclude other or all host computers 104 a-b electronics modules 33 a-bfunctions or just selected functions, like the display assembly 217function; such that other functions are run on an adjacent system incommunication with a host computer 104 carried elsewhere on body of theuser or located elsewhere as part of the telecom network and system 105.FIG. 22b is a see-through perspective illustrating the display assembly217 located on the head of the user. The display assembly 217 isconcealed by the user's natural skin and hair, or what could be ahairpiece and/or skull cap. Alternatively, the module may be implantedin a cannular fashion. In this example, one end of each display assembly217 a-b may comprise a corresponding near eye display 220 a and 220 b(referred to collectively as 220 a-b, see FIG. 22b ). The near-eyedisplays 220 a-b in the present example may comprise either aholographic display or LED/OLED display. The displays 220 a-b may beretracted and/or stored in the display assembly housing. The displaysmay extend outward from one end of the display assembly in front of theeyes of the user. On the other end of each display assembly an earbud138 a-b is extended out from the display assembly to the ears of theuser. The display assembly may be supported in any of the methodsdescribed in FIG. 2a-f or by the skull cap previously discussed in thisspecification. In the present example, the display assembly is connectedby display assembly support brackets 221 a-c. In the present example,the assembly 217 a includes an electronics module 33 a that connects tobrain activity sensors 81 a-c and 81 d-f that surrounds a specificneural area 219 that provides feedback to the electronics module. Theelectronic module communicates with and powers the implanted brainactivity sensor(s) 81 a-c and 81 d-f via communication lines 206 et almade of clad wire, fiber optics, or wirelessly 76. Additionally, in thepresent example assembly 217 a includes an electronics module 33 a thatconnects to a non-interference field-of-view support assembly 218 forfacial and panoramic imaging that communicates with the electronicsmodule.

Ear buds and displays worn by the user provide feedback to the user fromvarious internal and external sensor systems worn consistent with thepresent invention 100. The electronics module includes a transceiverthat connects wirelessly to a WIFI network that may include a remotehost computer server system 105. Power for the display assembly isprovided by battery. The battery may be located in the display assembly,or from a wire running from the module to the battery carried by theuser, such as a belt worn battery pack.

FIG. 23a-f illustrates an alternative embodiment of an implantableextendable and retractable electronics display assembly 217 that may beused with the present invention. FIG. 23a is an exterior perspectiveview of a user 101 wearing a left and right display assembly 217 a-b.FIG. 23b is a frontal sectional view showing the location of the displayassembly 217 a-b implanted in the head of the user. FIG. 23c is sidesectional view showing the location of the display assembly 217 a-bimplanted in the head of the user. In the present example, each near eyedisplay 220 a-b of each display assembly 217 a-b is shown in theextended position in front of each of the eyes 225 a-b of the user 101.Also, in this example, a portion of the frontal bone 223 of the skull 13is removed and the assembly is implanted. The implant may be implantedin the forehead or temporal area of the head. Surgical procedures forimplanting such a device are known to those in the medical profession.The assembly is anchored to the skull just as in a method similar tothat commonly used in skull replacements. Plastic or metal material istypically used. Alternatively, the module may be implanted as asub-dermal implant just beneath the skin. FIG. 23d is a front sectionalview showing the components that comprise the implantable displayassembly 217. FIG. 23e is a front sectional view showing the componentsthat comprise the implantable display assembly 217. The display assemblymay include an automated or manual adjusted swivel 231 for positioningthe display 220 in front of the eye of the user. The display assemblymay include firmware 207, circuitry 26, and be connected by electricalcable 206 or wirelessly via transceiver to the host computer 104 and/orelectronics module 33. The module may be controlled by brain activity,voice, manually, or common interactive device control methods. Thecannular opening 227, from which the display retracts, is a small slitin the skin. The opening 227 in the implantable cannular housing 16 ofthe display assembly 217 may include a sleeve or boot for thesurrounding skin 157 to grow into. Special medical care and techniquesare known to the medical profession, and body art profession, and arepracticed in order to get proper skin growth around but not covering theopening in order to avoid infection. FIG. 23f is a diagrammaticsee-through axonometric schematic with arrows 222 indicating the motionof the extendable and retractable near eye holographic display. Steppermotors 226 are used to extend and retract the near eye holographicdisplay. In FIGS. 22a-b and FIGS. 23a-f the near eye display 220 isshown in the un-retracted state in which the user is employing thedisplay for viewing. The display is see-through and allows for overlaidgraphics and imagery over the real-world scene in the surroundingenvironment 160 in an augmented reality (AR) manner. Augmented andtelepresence overlays may be displayed on the display in accordance withthe system 100 that comprises the present invention. A holographic neareye display system is shown in the present example. However,alternatively, a near eye display that incorporates e-paper, LED, orOLED displays like those shown in FIG. 8a-8c , FIG. 10c , FIG. 18, orother via other HMD arrangement may be incorporated in the presentinvention. Still alternatively, an EMD near eye display, as shown inFIG. 1 and FIG. 19 may also be incorporated for AR applications. It isanticipated that eye tracking and camera systems known to those skilledin the art will also be incorporated with the near eye displays withoutdeparting from the scope of the present invention 100.

FIGS. 24a-f are illustrations of a user 101 mountable integrated cameraand display assembly 148 consistent with the present invention 100. FIG.24a is a perspective drawing the mountable display system 148 affixed tothe thumb 232 of the user. FIGS. 24b-24d show different embodiments ofthe thumb mounted assembly 148. The assembly 148 preferably comprisese-paper, LED, or OLED displays like those illustrated in FIGS. 10a-cand/or 11. However, it will be apparent to those skilled in the art thatother display technologies may be integrated without departing from thescope of the present invention. And it will be apparent to those skilledin the art that optionally the display and camera may be separated andonly one or the other may be housed on the assembly 148 and the otherworn elsewhere as a separate assembly. In both FIGS. 24a-f the assembly148 comprises an integrated camera and display array 47 or 156 thatforms the outer surface of the body or body covering as depicted inFIGS. 10a-c and/or 11.

In FIG. 24b the assembly 148 includes a display boot 233 that slipssecurely onto the thumb. The boot 233 includes an integrated display andcamera array 156 that fits over the finger nail and communications linesand power cables 206 that runs from the array that transverses throughthe boot and is in communicating relationship to an inductive electricalreceiver pad 191. The electrical charger receives electrical current viainduction system integrated into the steering wheel of an automobile onwhich the hands of the user are positioned as depicted in perspectiveillustration shown in FIG. 24f FIG. 24c is a sectional diagram ofanother embodiment of the user 101 mountable integrated camera anddisplay assembly 148 in which the assembly is part of a prosthetic thumb235 worn by the user. FIGS. 24b and 24c include an integratedtransceiver for receiving and sending imagery and receiving controlsignals and electrical arrangement for receiving electrical power. InFIG. 24c assembly 148 comprises a prosthetic electronics unit 236 thatconnects to the nerve endings of the end of the thumb and controls thumbmovement and a camera and display with transceiver and battery 21electronics unit that controls camera and display operations.

FIG. 24e is a diagram of a user with very small electrical power and/ordata cable 206 run between material that comprises a sleeve 234 thethumb 232 fits into. The power and/or data cable runs from the assembly148 to the host computer 104 and/or electronic module 33. Assembly 148may be integrated into the sleeve or adhered to and stuck onto the topof the fingernail of the user. The material that comprises the sleeve234 may be a tattoo sleeve or a glove that slips on and off by the user.Alternatively, the sleeve may have a sticky side, like an adhesivebandage with an adhesive side, which adheres to the body of the user onone side. In such an instance, a flesh colored material that blends withthe skin color of the user may be incorporated for cosmetic effects tohide and insulate the very small insulated data and power cablessandwiched between the outward facing and inward facing material of thebody of the user. Finally, FIG. 24d illustrates and embodiment in whichthe thumb 232 mounted integrated camera and display assembly 148receives and sends electrical power and data over a small cable 206implanted under the skin of the user. The data cable may comprise afiber optic data cable. Obviously, various arrangements andconfigurations of components in FIGS. 24a-f may be utilized withoutdeparting from the spirit of the invention. In FIGS. 24a-f the thumbmounted assembly 148 may either communicate with the host computer 104and/or electronic module 33 to contribute to the operation of thepresent invention 100.

Sheets 23-35 (FIGS. 25-37 c) describe the processes and methods enablingthe invention. The subsystems, assemblies, and components previouslydiscussed in the specification are applied in as processes and methodsthat make possible the applications of the present invention 100.

FIG. 25 is a schematic diagram that illustrates the overall scope andmethod of interaction of the present invention. FIG. 1 and FIG. 25 worktogether to illustrate how the present invention operates as a userportable integrated life logging and enhanced memory assistant (LLEMA)system 100. The system 100 comprises a portable host computer system 104with a user interactive command and control module 107, an internal andexternal sensing, recording, monitoring, and data logging module 109,and correlation module 111. Modules 109 and 111 include hardware andfirmware for simultaneously sensing, storing, and correlating signaturesrepresenting the local surrounding environment 160 about the user 101with signatures representing the internal physiological environment 1 ofthe user 101. The surrounding environment may include the periphery ofthe user, such as the face of the user. The primary internal signatureslogged represent activity of the central nervous system 12, specificallythe neurological activity 165 of the brain 167 of the user 101. The sameuser interactive command and control module 107 also operates to commandand control querying for the user 101 based on data and informationsensed and logged by the system 104.

System 100 may include an optional telecommunications system and network105 and optional remote computer server system 106. Thetelecommunication system and remote server may be located in the localsurrounding environment 160, another part of the world, or anywhere inthe universe that has a compatible communication system that hastransmission between the portable computer 104 and remote server system106. The server system may comprise a single or group of networkedservers. The remote computer server 106 system will typically comprise astationary computer server used to store and process offloaded data fromthe portable computer 104. Offloading functions from portable computer104 to computer 106 facilitates reduction in size and volume of theportable computer 104.

FIG. 26a describes the process 200 for implementing the system 100 and104 according to the present invention. Step 1 239 is to: a) Inputsignatures (i) from physiological and biometric sensors representing aperson's internal state of being at a given time and place; and (ii)simultaneously input signatures representing the external environmentpresented to the person (i.e. via audio-visual sensors); and b) operatea computer to correlate the internal and external signatures into ahistorical relational database. Step 2 240 is to: a) Store thehistorical relational database into the memory of a computer; b) Querythe historical relational database to find correlations between currentinternal and external signatures; and c) Read in historical informationrecorded in the host computers memory into the being using i) sensoryinput and/or ii) implanted devices to enhance the being's thoughts andmemory. Step 1 enables Step 2 by providing internal and externalhistorical data and information that is then logged, processed,recalled, related, and correlated by computer 104, and optionally andadditionally computer 106. Arrow 241 indicates that once Step 1 isaccomplished then Step 2 may commence. Arrows 242 a and 242 b illustratethat Step 1 and Step 2 are related independent continuously loopedprocesses that may run on computer 104 and/or 106 once a baseline ofinternal and external signatures is established.

FIG. 26b is a diagrammatic representation illustrating the process 200described in FIG. 26a for a memory enhancement system according to thepresent invention 100. Consistent with Step 1, in this example theportable AMR system records a brain activity signature of a user 101 ata given time and place when the user of the system 100 sees a cat in thesurrounding environment. The brain activity signature indicates activityin the thalamus lateral geniculate nucleus area which decodes signalsfrom the retina of the eyes of the user. If the brain activity signatureis operated upon by a computer to derive a low-resolution image 243 ofthe cat (See Yang 1999 referenced previously). The brain activity dataof the user indicates a cat of only sufficient detail, similar to whatthe user is able to imagine in his or her mind. Based on the time andplace the brain activity of the cat occurred in the brain of the user,computer 104 performs a query to search for related video of the catrecorded by video sensors worn by the user that took imagery of the catat the same time and place when the brain activity of the cat occurred.The logged video imagery of the cat from the system 100 shows anincreased detailed image 244 compared to the lower detail image 243rendered from just the brain signature data. The historical database isoperated upon using pattern matching techniques comparing the AMR datato the historical video database which shows a more detailed image 244of the cat. The host or a remote computer analyze and compare the AMRdata to video data and confirm it is the cat belonging to the user. Aspecific neuron, group of neurons, or pattern of neural activitypreviously correlated to a specific cat, named “Pussy”. The user isseeing or thinking about such that the query process searches for thespecific cat in the database. If the query results in finding thatspecific cat then associated data defined by the user about the cat iscalled up by the computer 104. Using the just described method 200 theuser can command the system 100 to find a more detailed image of “Pussy”than he can remember in his mind's eye by calling up a portion ofimagery recorded in the relational database that was previously loggedand filed using the system 100. Preferably the findings of the query arepresented to the user by an interactive input device worn or implantedinto the user. For instance, the high-resolution imagery related to thecat is may be transmitted to the left and right displays of the HMDsystem and the audio of the cat is transmitted to the left and right earbuds of the HMD that the user is wearing.

FIG. 27 is a block diagram that illustrates a method which comprisesStep 1 239 generally described in FIG. 26a , of operating the portableinteractive life logging input and storage system that is part of system100. In FIG. 27 the first phase of operation of the data logging systemis illustrated. Step 1, Phase 1, generally described in FIG. 26, and nowspecifically described, comprises the operator turning on the wearablehost computer system 104 to activate the internal and external datalogging system. Step 1, Phase 2 is the logging system recording brainand video imagery. A portable Atomic Magnetometer Sensor Array MagneticResonance (AMR) Imaging System, referred to herein as an AMR system,transmits an image signature of brain activity to pre-processors ofcomputer system 104 and/or 105. Simultaneously, a portable panoramicvideo recording system transmits image and audio signature of thesurrounding environment to pre-processors of computer system 104 and/or105. Preprocessing may include the normalizing the signatures into acommon readable language within the host computer by performingtranslational computer processing operations of data output from thevarious sensors, rotation, translation, scaling, brightness, contrast,time, date, and geo-spatial tagging of the data. In Step 1, Phase 3 thebrain patterns and video imagery are processed and stored into memorysegment folders in the computer 104 and/or 105. At least one of thecomputers 104 or 105 includes a Cognitive Auto-Associative NeuralNetwork Based Search Engine that performs computer operations toidentify and report corresponding correlations between the internal andexternal data sent from the sensors. Image and brain signatures arenormalized, correlations are searched for, and above-the-thresholdrelationships found of neural to audio and image correlations are loggedand recorded in the historical database by subject, time, date, andgeospatial coordinate information. Similarly, the auto-associativeneural network based search engine may also be trained to identifyneural correlations of consciousness from various other types sensorinput data which are logged into the memory folders of the computer 104.Arrows in the drawing 241 indicate that once each step is completed thesystem 100 and/or 104, and the user 101, preferably proceed to the nextstep in the process. An arrow 242 a indicates a repetitive loop in theprocess which the system 100 and/or 104 preferably operates uponcontinuously when the system is activated.

FIG. 28 and FIG. 29a-b provide example diagrams of panoramic imagery andbrain activity imagery representing the same subject matter that may belogged by a user 101 into the computer system 104 or 106 who is bearingthe present invention 100. While these diagrams are shown graphically tofacilitate understanding of the present invention, it will be evident tothose skilled in the art that signals and/or images derived frominternal and external senor signals may be translated into and/orrepresented as computer language for computer processing and archivalpurposes.

FIG. 28 provides an example composite image frame of undistortedpanoramic imagery 246 a-b taken at Time and Place #1 248 by a panoramicspherical field-of-view (FOV) video camera system. Several underlyingrectangular shapes are shown to indicate that the panoramic camera isrecording consecutive rectangular video image frames at Time and Place#1 to the nth 249. The panoramic camera faces outward from the user'sbody, and may be mounted in or on the user in a variety of ways whichwere highlighted in FIG. 2a-f and discussed earlier in thisspecification. In this instance, barrel distortion of the image framehas been removed by using a special fiber optic arrangement, as shown inFIG. 4, such as the one referred to in the prior art by the trademarkFibreye™, to remove barrel distortion. Then the two back-to-back images246 a-b are processed by computer 104 and/or 106 into a single videoframe where the images 246 a-b are placed adjacent to one another in anatural continuous panoramic scene format for input into the hostcomputer 104 or 106 of the present invention. Side A is taken by onefisheye lens and Side B is taken by the other fisheye lens. Each fisheyelens (via the associated camera) records a hemispherical FOV image withgreater than or equal to 180-degree field of view coverage that may bestitched together to form a continuous spherical panoramic scene.Alternatively, distortion may be removed and the images stitchedtogether optically, as in the present example, or by computer imageprocessing. Within the frame, the shape of a “Human” 255 a and a “Cat”255 b are called out as representations of objects that the computer 104and/or 106 may use image recognition processing on to identify as a“Human” and a “Cat” based on historical information in the computer 104or 105 data base. And furthermore, the computer 104 and/or 106 may beused to find relationships between the image of the “Human” and “Cat”and brain activity as illustrated in FIGS. 29a and 29 b.

FIG. 29a is a drawing that represents an image of a latitudinalsectional brain scan 251 from an AMR. The oval shape represents a brain167 scan of the user 101 taken at specific instance in Time and Place #1251. Several underlying oval shapes are shown to indicate that the AMRis recording consecutive brain scans at Time and Place #1 to the nth252. The irregular shapes within the oval represent brain activityassociated with the subject and activity as perceived by the user at agiven time. For instance, in the present example the CP of the user is a“Cat”. “Cat” brain activity signatures resulting from the user observingthe “Cat” in the surrounding environment are operated upon by computer104 and/or 106 using image recognition software and/or firmware toidentify the irregular shapes that represent the brain activity of the“Cat” NCC's 166 based on historical information in a computer 104 and/or106 database. Additionally, computer 104 and/or 106 are operated to findrelationships between the brain activity data and informationillustrated in FIG. 28 and the panoramic video imagery data andinformation illustrated in FIG. 29a-b of the “Cat”. FIG. 29b is adrawing that simply shows a different method of reading out the brainactivity during the same timeframes as shown in FIG. 29a , but in athree-dimensional voxel image format. In FIG. 29b the brain activitythat meets a certain hit criteria and threshold level is displayed asthree-dimensional voxels at Time and Place #1 253. Multiple samplings(not shown) of the voxel imagery over time may be recorded and processedin computer 104 and/or 106 to build neural correlates of consciousnessfor a Time and Place #1 to the nth 254.

It will be extrapolated by those skilled in the art that FIGS. 28 and 29a-b teach that other sensor signatures may also be operated on to builda body of knowledge in the computer 104 and/or 105 to define objects andactions by a plurality of cross-correlations derived by analysis ofsensor data stored and processed in the computer 104 and/or 105. Othersensor systems that can provide input data include sub-vocalizationsystems, audio recording and processing systems like ambisonic audiosensor systems, touch sensitive system, and laser rangefinder and targetdesignation systems. Additionally, informational search engine databaseslike Google®, Wikipedia®, and so forth can provide data input tocomputer 104 and/or 106. Social Network sites, like Face Book®,Twitter®, and so forth may also provide data input to computer 104and/or 106. Additionally, a telecommunication and interactive devicethat the user operates may be data-mined for input into computer 104and/or 106 to assist in identifying conscious percepts that defineneural correlates and relationships to items in the surroundingenvironment and the brain. Data-mining may be conducted manually by theuser, or via automated search engines configured to operate without theuser's specific guidance, based on user focus, or other criteriapre-programmed into the system 104 and/or 105. Information defined onthese sites both overtly and through context may be used to form astrong body of information to correlate with brain activity patterns,neural activity and associated signatures.

FIG. 30 is a diagram illustrating the method of constructing neuralcorrelates of consciousness (NCC's) from internal and external sensordata recorded from and about a user 101 in the present invention 100.For instance, signatures of brain activity are correlated with audio andimagery recorded in the surrounding environment about the user 101 at agiven Time and Place #1 to the nth 252. In other words, the computer 104and/or 106 is trained to recognize that certain brain cell activitycorresponds to certain objects and activities in the environment. Toaccomplish this recognition computer 104 and/or 106 operate to identifyNCC and construct NCC Correlation Tables 250 that are historically andstatistically sufficient to define a threshold relationship betweeninternal and external signatures central to the CP of the user at agiven time and place. Digitized brain activity imagery 257 in additionto digitized surrounding scene imagery 258 (including peripheral imageryof the body) is stored by and operated upon by computer 104 and/or 106to derive the NCC Correlation Tables 250. The correlations are thentranslated into computer code that represent a NCC that define specificsubject matter or activity related to that CP. The correlations and/orrelationships that form a NCC database 250 may be stored in computermemory. The database 250 is represented in machine language or computercode. Computer 104 and/or 106 used the computer code and associatedalgorithms to operate upon and query the database 250. Various portablebrain activity sensing systems of a type that may be used in the presentinvention to record brain activity of a user at a given time and placeaccording to the present invention are described previously thebackground of the invention and earlier in this specification. Variousportable panoramic camera systems of a type that may be used in thepresent invention for recording audio-visual representations of thesurrounding environment about a user at a given time and place aredescribed in the background of invention and earlier in thisspecification.

Still referring to FIG. 30, once a subject or an action has beenidentified as the CP the brain activity of Person 1 the NCC's for thatsubject matter or activity are constructed as a correlation table oralgorithm 250 in the computer 104 which defines various sensorsignatures that mean the same subject or activity between internal brainactivity and external surrounding environment sensor system data andinformation. The subjects and activities logged can be expressed by aword or words groups, numbers or number groups, in whatever language iscompatible to the users, be they man or machine. The rectangular boxeswith numbers graphically indicate computer code derived by operatingsystem 104 which is receiving and processing Person 1's brain activityat a given Time and Place 1 251 that has a CP of the subject Cat. Thecomputer code 257 represents Person 1's brain activity at a given Timeand Place 1 251, the computer code 258 represents Person 1's surroundingenvironment at a given Time and Place 1 251, and computer code 250represents the NCC Correlation Tables derived by computer 104 at a givenTime and Place 1 251. This relationship can be expressed as Person 1'sA's brain activity of the Cat=C, Person 2's A's imagery of the Cat=B,and the resultant normalized NCC Correlation Tables where B=C. Forinstance, language may also be composed of computer language such asC++. The computer 104 database preferably comprises a database with metatags and metadata to locate other data in a search engine that performstranslation correlations between persons or machines as theycommunicate. Databases like those in the CALO and neural network systempreviously described in prior art may be incorporated to assist intranslation between users. Typically, the correlation tables andalgorithms will comprise look-up tables in the computer which relateuser neural correlates to logged video. The translation key equatesPerson 1 brain activity patterns to images and audio, with words thatare formed by the computer 104 and/or 106 operating upon look-up tablesin the computer 104 that equate to similar representations, such aswords. Various normalization, correlation systems, correlation tables,and translation keys are widely known and used in the computer industryso will not be described in any more detail in this specification. It isanticipated various search normalization, correlation, transcription,and translations systems will be used in embodiments of the presentinvention. Obviously, because the brain is a dynamic organ, updates ofthe correlation tables of Person 1, and correspondingly, machine 104and/or 106 will be required periodically to main current and relevant.The correlation tables and algorithms of Person 1 may reside in computer104 and/or 106, depending on the design of the system 100.

FIG. 30 shows a latitudinal cross section of the brain on the left andan image frame on the right at Time and Place #1 251. The FOV of eacheye of the user is indicated by solid circles 255. While dashed circlesindicated adjacent hemispherical FOV images captured by the panoramiccamera system worn by the user. The two hemispherical images are subsetin a HD image frame 256. A ROI image sensor system may be used toidentify, track, and sample out the desired portions of the panoramicscene in the FOV of the user, here indicated by solid lines. Distortionmay be removed optically (i.e. Fibreye™) or by image processing. Variouscamera and sensor arrangements discussed throughout this disclosure arepossible. Objects may be designated as neural correlates based onvarious actions by the user. For instance, as the AMR system recordsbrain activity and the panoramic video system records audio and imagerya target tracking system monitors the focus and gaze of the eyes of theuser on a particular subject in the surrounding environment. The targettracking system marks the location of the subject on the recorded videothat the user was gazing upon. The location data of the subject gazedupon is then stored with the imagery in a database. A target trackingsystem of a type that is incorporated into the present invention thatmay be used for determining gaze and target designation within thefield-of-view of a user according to the present invention is describedpreviously in this application. As previously described in thisspecification data and information from the tracking system may beoperated upon by computer 104 and/or 106 to identify the CP that theuser if focused upon. Similarities and differences of brain activity aremeasured and recorded as various subjects are observed and activitiesare accomplished. Pattern analysis is conducted, and correlations aredrawn, via computer processing between subjects focused upon andactivities being accomplished, and the user's brain activity. Thesesimilarities and differences are measured and recorded. The resultingneural correlates are established based on the strength threshold levelset by the user operating the correlation system. Once correlatesbetween brain activity and subjects and actions are identified therelationships are logged into the rule sets data base of the loggingsystem. Repeated recording of very strong excitation of a certain neuronin the brain when the same subject is observed in the surroundingenvironment can be used to establish a strong neural correlate. Weakcorrelations may be discarded. As illustrated in FIG. 28, when a userobserves a “Cat” with his eyes in the surrounding environment, a certainneuron or neurons 164 in the brain repeatedly fire while the “Cat” isobserved and is the CP of the user as illustrated in FIGS. 29a and 29b .Additionally, associated neurons firing and the sequence they fire inthe brain that are related to subjects and activities over time in thesurrounding environment or to a given thought by the user are recordedby the logging system and provide additional data for building even moresophisticated neural correlations. These correlations are recorded intoa database 250 for later reference. A system of a type that isincorporated into the present invention that may be used for processing,storing, pattern analysis, and determining the strength of thecorrelation of incoming sensor data for building correlations accordingto the present invention has been described previously in thisapplication. Multiple sensor correlations may be used to build strongcorrelation factors and database tables that reduce the risk inmisidentifying a subject, object, or an activity. For instance, unlesscomputer is able to achieve a neural correlation based on sensoranalysis data of 95%, the “Cat” will not be identified as the “cat”belonging to the user. In our present example factors confirming the catbelongs to the user is historical data that shape, sound, color, colorpattern, size, time, and place associated neurons activated in the brainalong with the video recorded that provides image shape and color,geospatial coordinates where the image was derived, time the image wasderived, and audio of what my cat sounds like when in meows are similar.If the composite standard deviation yields a confidence level on allthese factors below 95% then computer 104 and/or 106 will notify theuser 101 that this not the non-allergenic cat belonging to the user. Ifby deduction, the computer operates to determine it is not the catbelonging to the user, the computer will display a text message to theuser that he or she should depart the area before he or she has anallergic reaction. Computer 104 and/or 106 may be operated by a user oradministrator of the system 100 to train the computer to recognize NCCs.Or computer 104 and/or 106 may programmed to autonomously derive NCCsfrom brain data to other sensor data during life experiences. In thisinstance, over time correlations will be drawn automatically andautonomously by the computer using the body of evidence built up throughanalysis of brain activity and other internal and external sensor systemdata. In either case, the established database may be modified and addedto by a user's body of experience and in order to increase the strengthof the correlation and resulting NCC database 250.

FIG. 31 is a diagram illustrating normalization of data and building atranslation table of brain activity between two different users: Person1 and Person 2. Brain activity in Person 1 and Person 2 are different,indicated graphically by the different shapes representing brainactivity imaged in Person 1 101 a and Person 2's 101 b's brain at Timeand Place 1 251 or Time and Place 1 to the nth 252 by the AMR system.However, the differing CP of the brain activity in Person 1 and Person 2is still representative of the same subject matter, a given “Cat” forexample that is being perceived in the environment and/or thought aboutin the brain. Once a subject or an action has been identified as the CPthe brain activity of Person 1 and Person 2 the NCC's for that subjectmatter or activity are constructed into a translation key 259 in thecomputer 104 which defines various sensor signatures that mean the samesubject or activity between Person 1 and Person 2. The subjects andactivities logged can be expressed by a word or words groups, numbers ornumber groups, in whatever language is compatible to the users, be theyman or machine. The rectangular boxes with numbers graphically indicatecomputer code derived by operating system 104 which is receiving andprocessing Person 1 and Person 2's brain activity at a given Time andPlace 1 251 that has a CP of the subject Cat. The computer code 257represents Person 1's NCC state at a given Time and Place 1 251, thecomputer code 258 represents Person 2's NCC state at a given Time andPlace 1 251, and computer code 259 represents the translation codederived by computer 104 state at a given Time and Place 1 251. Thisrelationship can be expressed as Person 1's A's NCC of a Cat=C, Person2's A's NCC of a Cat=B, and the resultant translation tables where B=Conce the NCC of Person 1 and Person 2 are normalized with one another.For instance, language may also be composed of computer language such asC++. The computer 104 database preferably comprises a database with metatags and metadata to locate other data in a search engine that performstranslation between two persons or machines as they communicate.Databases like those in the CALO and neural network system previouslydescribed in prior art may be incorporated to assist in translationbetween users. Typically, the translation tables will comprise look-uptables in the computer which relate user neural correlates to languagetranslation keys, so Person 1 and Person 2 can communicate between oneanother. Various normalization, correlation systems, correlation tables,and translation keys are widely known and used in the computer industryso will not be described in any more detail in this specification. It isanticipated various search normalization, correlation, transcription,and translations systems will be used in embodiments of the presentinvention. Once a common NCC translation table is constructed for Person1 and Person 2 then a translation key is built in the computer 104 thatallows Person 1 and Person 2 to communicate with one another asillustrated in FIG. 31. Obviously, because the brain is a dynamic organ,updates in the translation tables between Person 1 and Person 2, or forthat matter, machine 104 a and 104 b will be required periodically tomain current and relevant. The translation key may reside on Person 1and/or Person's computer 104 and/or 106, depending on the design of thesystem 100. The translation key equates Person 1 and Person 2's brainactivity patterns and/or images, or audio, with words that are formed bythe computer operating upon look-up tables in the computer 104 thatequate to similar representations, such as words. For instance, as shownin FIG. 31, when Person 1 thinks about feeding his cat his brainactivity reflects a certain brain activity pattern. The brain activityof Person 1 is analyzed by the computer 104 and/or 106 using the alreadybuilt correlation tables described in FIG. 30 and the thought isdeciphered by the computer 104 and/or 106 into the words “Feed the cat”.Even though the brain activity patterns for the same subject matter andactivity are different between Person 1 and Person 2, the areas ofactivity are correlated to certain subjects and activities, which may beuniversally and similarly defined between different users by thetranslator system. Types of brain activity patterns that include brainregion and neural activity, location, and interaction at Time & Place #1to the nth. The message “Feed the cat.” may be transmitted into the textlanguage or spoken language as required using translation tables. Textand voice translation systems and associated tables are widely known inthe computer industry. But in contrast to conventional word translationsystems, in the present invention brain activity patterns, images, andaudio representations between two persons are translated into a commonlanguage, which may be conventional text or an audio representationrecorded in the computer 104 and/or 106. This facilitates man to machineand machine to man communication. Continuing with our present example,the words, “Feed the cat.” are transmitted over telecommunication systemand network 13 from Person 1's computer 104 and/or 106 to Person 2'scomputer to communicate the message. The message may be transmittednon-verbally between the two individual users of the system just bythinking of the action. The words are then displayed in text on a HMD orEMD (i.e. a contact lens eye display) of Person 2. Alternatively,instead of text, an audio signal may be output using a voice synthesizerto Person 2. Still further, another computer, machine, or robot mayreceive the textual or acoustic message communicated from person 1. Inthis manner person 1 and person 2 may communicate non-verbally viamachine interface. When the data logging system is activated computer104 and/or 106 is programmed to automatically and continuously constructand update correlation and translation tables based on sensory input anduser commands. With respect to building translation codes, a user mayuse a computer menu and input device to specify that the computer 104and/or 106 build a translation key or table that allows him or her tocommunicate with a particular person or machine. Sensory input mayinclude brain activity sensors, panoramic sensors, or other input fromvarious input devices. In this manner computer 104 and/or 106 builds adatabase for user communication based on brain activity that isassociated with his or her experiences in and with the surroundingenvironment.

FIG. 32 is a block diagram that illustrates method which comprises Step2 240 generally described in FIG. 26a of operating the portableinteractive query and memory enhancement method that is part of system100. As illustrated in FIG. 32, once the data logging subsystemcomprising computer 104 and/or 105 with correlated and translateddatabases has been established the interactive portable memoryenhancement subsystem operates to access the information for memoryenhancement operations. Metadata referencing related subject matter seenin brain patterns and video of related indexed historical data in filefolders such as those described previously in this application allowsfor instantaneous feedback to the user by the auto-associative neuralnetwork. High-speed telecommunication systems and computer systems areincorporated to do what is called “fast fusion” of informationcommunicated. The objective is real-time processing and feedback ofcomputer 104 and/or 106 to the user of the system 100. Memoryenhancement operations will typically be defined by the person who wearsthe system selecting rules from menus that are part of the firmwareeither in the host or a remote computer. Interactive user menus may besimple yes or no, or more sophisticated noun, verb, sentence or graphiclike. User menus may be manual or set up to be automated requiring noconscious decisions, depending on how the user sets the defaults ofcomputer system 104/105. At any point after at least some portion of thecorrelated database has been formed in Step 1 a translation databasewith another system that has a different representation of subjects andactivities may be constructed in a manual or automated manner. Thetranslation key is also introduced and stored as a portion of thedatabase of computer 104 and/or 105. The translation key/database isqueried when interacting with a human that speaks or machine thatoperates in a different language. In this manner machines and beingswith different languages can communicate. The user bearing the datalogging and memory enhancement system 100 will typically set theparameters and command system 100, but this may alternatively be adifferent person. In such instances the person who operates the systemand its associated menus does not have to be the user. For instance, aremote operator of system 104 or 105 could control information flowinginto a user's input devices, such as a HMD. That operator may be anotherperson or a machine.

FIG. 32 elaborates on Step 2 240 of FIG. 26a . FIG. 32 is a systemsdiagram that illustrates the user 101 interactive portable memoryenhancement portion of the invention 107. In Step 2, Phase 1 is theoperator turning on the host computer system 104 and activating thememory enhancement system. Step 2, Phase 2 is a stimulus in theenvironment, or a thought and/or action generated by the userstimulating the mind and causing brain activity. In Step 2, phase 3 thehost computer receives the signature from the AMR of the active brainactivity and queries the correlated databases for matches between theactive brain activity and the historical brain activity databasepre-associated with historical information. The brain activity mustregister a certain level of measurable and recognizable focused brainand other sensor activity before a query or command based on thatactivity is operated upon by computer 104. Once the requirements are metthe information is operated upon in the computer 104 and/or 106 to andthe historical database is queried. In Step 2, Phase 4 the matches arepresented to the user and then in Step 2, Phase 5 the user chooses whatmatches to activate and act upon. However, a system administrator oroperator may be employed to set-up the system 100 and/or 104.Additionally, the system administrator or operator may preselect menuoptions for a user. Optionally, the system administrator or operator mayor may not be the user of system 100 and/or 104. In the present contextthe “system administrator” or “operator” is referred to a person ormachine who establishes the parameters and/or programs the system 100and/or 104. An operator can set rules using menus that pre-define whatinformation is transmitted for input to the user. The operator may ormay not be the user 101. And finally, in Phase 6, Step 2 the brain ofthe user is stimulated with matched information returned from the queryof a database. The database queried may comprise information derivedfrom building the NCC database (i.e. 257 and/or 258), the actual NCCCorrelation Table database 250, or external databases (i.e. socialnetworks and search engines on the Internet/GIG). Systems of a type thatare incorporated into the present invention that may be used forquerying correlated data logged into the system according to the presentinvention are described by U.S. Patent Application 20070124292 A1, byKirshenbaum et al., dated 31 May 2007, entitled Autobiographical andOther Data Collection System and U.S. Patent 2009/0196493, dated 6 Aug.2009, by Widrow et al. entitled Cognitive Method and Auto-AssociativeNeural Network Based Search Engine for Computer and Network LocatedImages and Photographs previously referenced the background of theinvention in this application. Arrows in the drawing 241 indicate thatonce each step is completed that the system 100 and/or 104 and the user101 preferably proceeds to the next step in the process. And arrow 242 bindicates a repetitive loop in the process which the system 100 and/or104 preferably operate upon continuously when the system is activated.

FIG. 33a-b is a table 247 a-b that illustrates a more detailed andintegrated description of the major component systems, their functions,and corresponding processes that comprise the life logging and enhancedmemory assistant (LLEMA) system 100 described in the present examplewhere the CP the user is focused upon is a “Cat” at Time 1. Rows are tobe read left to right and columns top to bottom. The data logging andmemory enhancement system 100 is tightly integrated because data andinformation from the sub-units contribute to the esemplasticity of thetotal system 100. To accomplish this detail it will be understood bythose skilled in the art that in FIG. 33a-b the terms system,sub-system, sub-unit, unit, module, and component may be referred tointer-changeably in certain instances without departing from the spiritof the present invention. The first row 401 of the table 247 a listsinternal sensor systems of the system 100 that operate to readout theirrespective content to the host computer 104 and/or 106. For instance, inthis preferred embodiment of the system the first row 401 indicatesthere is an fMRI system 421, panoramic video camera system 422,sub-vocalization system 423, and a target designation and trackingsystem with ROI and GPS 424 which comprise the sensors in this exampleof system 100. The second row 402 lists the type of signatures beingread out as data and/or information by the various internal and externalsensor systems. The signatures from the sensing systems may be read outin varying degrees, or preprocessed, depending on the design of theoverall system 100. As illustrated in the second row 402 the respectivereadouts output brain signature, panoramic imagery and spatial audio,sub-vocal system audio translation, and geo-spatial data andinformation. And as illustrated in the third row 403 the respectivebrain signature, panoramic imagery and spatial audio, sub-vocal systemaudio translation, and geo-spatial data and information are recorded andtransmitted to computer 104 for processing. Each of these outputs areinitially input into the host computer 104 for processing. Optionally,computer 104 may transfer the output information to computer 106 forassistance in processing data and information. The output data andinformation from the sensor systems may be simultaneously processedusing multi and parallel processing techniques know to the computerindustry to reduce latency. Now referring to row four 404, the fMRI unitreadout is recorded and transmitted to computer 104 from Time 1 to thenth when user thinks about, sees, hears, smells, or touches, a “Cat”.Likewise, the panoramic video camera system 422 reads out panoramicimagery and spatial audio that is recorded and transmitted to computer104. Rows five through seven, 405-407, indicate that brain activityinformation, imagery, audio, sub-vocal signatures and positional andgeospatial data is logged into computer 104 memory and stored in ahistorical database. Thresholds and rules are defined in computer 104firmware to filter out what sensor data and information is kept anddiscarded in the sensor data and information historical database.Likewise, the processing and determination of what data and informationto retain or disregard output from each sensor unit 421-424, whichcomprises the fMRI system, panoramic video camera, sub-vocalization,target designation and tracking units respectively, is accomplished bycomputer 104 and/or 106. It should be noted that not recording allinformation will limit later retrieval to only that which is retained inthe sensor historical database. The historical database organization,search, and retrieval design in the host computer 104 is constructed ina manner commensurate with the type of computer system, operatingsystem, and application firmware and software selected to accomplish thepresent invention 100. As illustrated in row five 405 the signatures arecorrelated in the host computer 104 artificial intelligence (AI) orAI-like system in a manner like that referenced previously in thisapplication in the background of the invention (i.e. Widrow et al). TheAI system correlates the common relationships in user time, location,subject matter, and activity that defines the subject “Cat” based onreal-time and historical data processing. As NCC's are identified thoseneurons, neural activity, and data and information in the historicalsensor database that supports derived NCC relationships is logged intocomputer 104 memory. Metadata retrieval systems and methods known tothose in the computer field may be utilized to quickly retrieve data andinformation that is logged into the host computer 104. Amulti-relational database may be established using any of the abovefolder heading (i.e. time, location, subject matter, and activity) aslong as metadata marks where the data is stored. Correlations betweensignature types are determined by computer 104. For instance, from Time1 to the nth computer processing by the AI system may indicate the user101 was in an environment in which correlations are determined betweenbrain activity in the user related to panoramic imagery and audio,sub-vocal, and geospatial data all related to the specific cat named“Pussy”. These relationships are then stored as metadata in the computer104 and/or 106 database.

FIG. 33b is a continuation of FIG. 33a . The first three rows of FIG. 3b, rows 408, 409 and 410, illustrate new sensor signatures are receivedby host computer 104 subsequent to the establishment of the relationaldatabase (DB) shown in row seven 407 of FIG. 33a . The new sensorsignatures may include user queries and commands. Only now, because arelational database including a historical database has beenestablished, are the new signatures received at Time 2 able to becompared to historical signatures and relationships recorded in the logof computer 104 and/or 106. Hence, as indicated by rows 408, 409, and410, is the newly received data and information at Time 2 to the nthable to now be operated upon by the host computer system to see if thenew data and information is the same, similar, different, or totallynew. And then, as indicated in row eleven 411, if significantrelationships are found between the old and new signatures then therelational database is updated, and if a response is required based on arule previously established by the user or an operator, the user isnotified, or an action is automatically taken. For instance, thesignature of a “Cat” prompts the computer to post a message on theuser's display, and/or a voice synthesized audio message is read to theuser notifying the user that the user may wish to take his allergymedicine before experiencing an allergic reaction to the “Cat” in thesurrounding environment.

Still referring to FIGS. 33a-b , in operation, tables 247 a-b indicatethat the system 100 is turned “On” to record internal and externalsensor data and information. The internal and external data andinformation is transmitted to the host computer 104 and/or 106 andoperated upon by command and control, correlation and/or translationfirmware. All internal and external sensor data to include head-and-eyetracking and global positioning 424, brain activity 421, voicerecognition or sub-vocalization 423 and video camera 422 readout dataand information as indicated in the four columns of the table 247 a-b istime, date, and/or geo-spatially stamped. Typically, the subjects in thesurrounding environment given the most attention as the ConsciousPercept (CP) by the user 101 will illicit the most brain activity andthe corresponding neural correlates of consciousness (NCC) areanalytically determined via computer 104 and/or 106 processing. Forexample, the internal and external sensors activated will indicate thatthe focus of the user is on a cat. When the user sees the cat, the brainactivity sensor system recognizes that the user is thinking about a catbecause certain neurons and brain activity takes place. Also, the useris likely to sub-vocalize the words, “A cat!” And also, because the useris surprised, he exclaims, “Oh, a cat!” verbally, out loud. Thepanoramic video camera system readout provides panoramic video to system104 for analysis. The target designation and ROI tracking unit 424sample the panoramic video feed for analysis. Because the user iswatching the cat with his eyes, the target designation system with ROItracking recognizes and locates the target, which is the cat, andprovides the geospatial location of the cat in the surroundingenvironment. Additionally, the panoramic video camera system 422 readoutprovides spatial audio which corresponds to the location of the cat ifthe cat makes noise and when the user exclaims, “Oh, a cat!”Additionally, because the user is allergic to cats, the breathing andheart rate of the user increases, which may be noted by biometricmonitoring sensors mounted on the user. All these internal and externalstimuli stimulate various parts of the central nervous system and brainof the user which is recorded by the brain activity sensor system.

Vocal and sub-vocalization signatures from microphones, electrodes, andvibration sensors is recorded and then compared with other historicalaudio and non-audio signatures. Sensor signatures are cleaned up andfiltered through the computer 104 in order to search for and determinerelevance and meaning. Noise from non-significant or sensor activity notrelated to the CP brain activity is filtered out. Magnetic interferenceis filtered out using computer processing techniques described in thereferenced prior art sensor fusion pre-processing systems to see if theymeet a certain threshold. Signature matches can be accomplished bycomparing incoming signatures from the sensors with previouslyestablished and recoded signatures that have been already defined asrepresenting the cat. Traditionally, in prior art, establishing thiscomparative database has been accomplished by an operator manuallytraining a computer that the pixels representing a cat image correlatesand corresponds to the audio signature that is translated into the word“cat”. But alternatively, signatures can be built on the fly as they arelogged in and processed in the present invention by comparing them withthe historical sensor database and NCC database, and then logging themin separately if they meet a threshold relationship that issignificantly different than previous signatures in the databases.Signatures are built on-the-fly or dynamically by building a body ofevidence among sensors signatures that have a certain relationship whichhave statistically significance.

Still using our “cat” as an example, we observe in AMR imaging that acertain neuron in a region of the brain shows increase blood flow when acat is focused upon by the user. The focusing of the user's eyes andears on the cat is determined by audio (ambisonic) tracking and imagerecognition and tracking systems that are part of the video loggingportion of the present invention. The video logging systems use patternmatching and other computer-based image processing methods to identifyaudio and images. The imagery from the video camera system will bematched up with the target tracking system, which incorporates a headand eye tracking system to see what subject is being observed and toprovide coordinates which will correspond to the user looking at thecat. Additionally, a laser rangefinder system of a type like thatdisclosed in the prior art is referenced and operates in the presentexample to further define the focus of the user. The laser rangefinderprovides the distance to a subject the user is focused on at a giventime. This distance, along with the GPS coordinates, assist in ROItracking and identifying a subject that stays within the FOV of thepanoramic camera. The range, location, and coordinates are operated uponby computer 104 pattern recognition software or firmware to identify thepattern as a cat and then log that specific pattern into the database asa cat. Additionally, other sensors' signatures are used to confirm theidentity of the subject cat, and also the subject's activity. Forinstance, the sub-vocalization system records a signature thatrepresents and is translated into the word “cat” when the user sees asubject cat. From these different internal and external sensors,correlations may be automatically established without human interventionby running operations programmed into the host computer. Thecorrelations also include a time stamp and GPS location and arereferenced to the raw data in the historical database which includesretained sensor database logs and the NCC database. When signatures arebuilt dynamically, manual training to define correlates of specificsubjects is not required. This is because the computer establishesneural correlations based on algorithms in the computer independentlybased on related internal and external activity and events that happensimultaneously in time (i.e. A certain neural activity and a certainimage pattern happening at the same time.). This is because as thedatabase grows, it gathers more information, which the system 100queries to identify new subjects and activities that take place in thefuture (i.e. Time 2 and Time 2 to the nth). In this way therelationships identified historically mimic human memory. The computeris thus trained to identify subjects and activities based on previousstored historical relationships stored in its database, much like humansgain from experience and store in the memory of the human brain.Furthermore, internal (i.e. AMR neural activity) and external (i.e.video: audio and imagery) correlation from sensors is bolstered when“hits” and “matches” are validated by the computer 104. The “hits” and“matches” to the relational database may be subsequent (i.e. discoveredduring in post processing) or sequential (i.e. discovered as newinformation arrives in near real time). Besides creating memory foldersin the host computers relational database that are logged using timestamps, memory folders logged using other categories, such as by subjector activity, are also possible. An example of an Internet search enginethe present computer system 104 could query is the Google® Internetsearch engine. Such a logging and retrieval system that is a type andintegrated into the present invention is U.S. Patent 2009/0196493, dated6 Aug. 2009, by Widrow et al. entitled Cognitive Method andAuto-Associative Neural Network Based Search Engine for Computer andNetwork Located Images and Photographs.

Sensors monitoring brain activity may be positioned to sweep the entirebrain or a portion of the brain. In particular a certain region of thebrain known to provide neural correlation for that subject, activity, orsensory perception may be targeted by sensors monitoring the brain. Forexample, in the New England Journal of Medicine article “WillfulModulation of Brain Activity in Disorders of Consciousness”, dated Feb.18, 2010, Martin Monti et al. [28], areas of the brain (i.e. activity inthe parahippocampal gyms and the supplementary motor area of the brain)are identified that show different activity when a user makes a “yes”response versus when the user makes a “no” response to a stimulus in theinternal and external environment. By translating brain patterns thatcorrespond to the user making “yes” and “no” determinations in his mindas he makes “yes” and “no” menu selections presented to him on ahead-mounted display or contact lens augmented reality display the usernon-verbally controls parameters within and input and output of the hostcomputer. U.S. Patent 2009/0196493, dated 6 Aug. 2009, by Widrow et al.entitled Cognitive Method and Auto-Associative Neural Network BasedSearch Engine for Computer and Network Located Images and Photographs.

Once data is logged in from the internal and external body sensors theinformation may be called up by the user to enhance his or her memory ormay be automatically input based upon predetermined rule sets the useror another person or machine has previously input into the hostcomputer. Referring specifically to the operation of software orfirmware on the computer of the in the body worn device s, the softwareincludes an operating system, atomic magnetometer, image and patternrecognition system, HMD, voice recognition, panoramic video, AI and/orAI like filing and retrieval, and telecommunications software orfirmware. Details of standard computer operations and interconnectingsoftware and firmware is already known to those skilled in the art anddescribed in detail in various prior art documents. In operation a fastfusion auto-associative neural network software and firmware of a typelike that described in Widrow is used to automatically query data in thememory folders in the data base of the host computer which is constantlyscanning the databases in response to queries for correlations toincoming internal and external sensory data. The information logged maybe referenced by artificial intelligent (AI) and AI like systems to forman enduring cognitive assistant for the user or another client in thepresent invention. An AI computer hardware and software of a type thatmay be integrated with the present invention is the Cognitive Agent thatLearns and Organizes (CALO), developed by SRI between 2003 and 2008[31]. CALO is a PC based cognitive software system that can reason,learn from experience, be told what to do, explain what they are doing,reflect on their experience, and respond robustly to the clients theuser specifies directly or through a user's repeated actions in using aCALO system.

The CALO system is integrated with the hardware and software andfirmware of the sensor pre-processing and Auto-Associative NeuralNetwork Based Search Engine for Computer and Network previouslymentioned through conventional computer system design widely known inthe computer industry. The CALO system is used to drive the command andcontrol portion of the invention. Computer system 104 may comprise oneor a plurality of systems, but must comprise at least one located on thesubject user or machine as described in the present invention.Translators may be used to link firmware and/or software applicationstogether or alternatively original code is written combining the codeinto a single application software or firmware. If brain activity,video, audio, and sub-vocal information meet a certain threshold ofsignature then the information may be selected or automatically input tothe user based on predetermined rule sets. Input means includeaudio-visual devices like a head-mounted display, electronic contactlens displays, and ear buds with small speakers. Alternatively, a menumay be presented to the user for selecting inputs. The system isprogrammed to display a list of yes/no items that correspond to audio orvideo files in the memory folders of the computer. In such an instancethe user may interactively review and act upon items in the menu inorder to select or not select items for input off the menu. Userinteractive command and control system 107 may include audio voicesynthesis and sub-vocal recognition systems 202, pattern recognitionsystems that recognize hand gestures, AMR brain activity patternrecognition, and the like. For instance, every time the user is around acat he may want to be prompted to take allergy medicine. When the systemrecognizes a cat in the surrounding environment the user may beprompted. Because the system is panoramic, it can look for things in oroutside the user's FOV if the user chooses to activate such a capabilitywithin the system. And/or still alternatively when a certain thresholdof sensory activity is met an injection of allergy medicine may beautomatically injected into the user's body by an adhesively wornmedicine pack located on the user's body that is in communication withsystem 100. Still alternatively, the system may be put on standby andonly log information in when a certain sensor reaches a certainthreshold. For instance, when a user's heartbeat reaches a certainpulse; when the brain activity of the user meets a certain threshold orcertain areas are active in the brain; or when a certain subject oractivity is sensed by the video camera with target tracking and targetrecognition.

FIGS. 34 and 35 are block diagrams that illustrate telecommunicationembodiments of the system 100. These embodiments of system 100 comprisea cloud computing arrangement for video logging and memory enhancementcomprising a local user portable host computer 104 and a remote hostcomputer 106. Optionally, as shown in FIG. 25, and more detail in FIG.34 and FIG. 35 the logging and enhancement system 100 incorporates atelecommunication system and associated telecommunications system andnetwork 105, like the global information grid (GIG), which includes theInternet. Some components of the system 100 may be placed in remotelocations apart from a user of the system such that it is unnecessaryfor the user to carry all components of the system 100. In certainembodiments of the invention, this is advantageous because carrying lesscomponents reduces weight, required electrical power, and componentspace for system 100 some of which must be borne by the user. Andfurthermore, it allows what is carried by the user to be carried lessconspicuously. The basic subsystems that are placed on or in the user'shead include a brain activity sensing system such as a portable AMR andpanoramic video sensor system. Additionally, and optionally, a voicemicrophone and/or sub-vocal recognition system is placed on the head orupper body of the user. Preferably, eye tracking and head trackingsensors are also located on the user's head or upper body. Othercomponents may be carried by the user other than on his head but in acommunicating manner to the sensor systems on his head or upper body,such as headgear 73, so that signatures may be transmitted to a remotedevice for processing. Transmission may be from the sensor systems borneby the user and in a communicating device to a portable electronicdevice borne by the user such as a PDA, cell phone, smartphone, reader,laptop, or other computer with wireless connectivity to an associatedtelecommunication system. The electronic device, such as the smartphone151, may be located in a backpack 133, belt pack, integrated intoclothing on ones lower body, or mounted in any suitable manner describedin FIGS. 1, 8, 14, and 15.

For instance, in the telecommunications embodiment shown in FIG. 34 theportable host computer 104 transmits internal and external sensor module109 data and information to the user 101 interactive command, control,communication, and computer (C4) module 107 that transmits data andinformation over a telecommunications system and associated network 105to a remote host computer server system 106. The user operatesinteractive input devices like a key pad, gesture recognition system,voice or sub-vocal recognition system to transmit command, control,communication, and computer (C4), panoramic video data, AMR brainpattern and activity data, and other sensor data and information. Thedata is transmitted from the portable host computer 104 includes has awireless modem and transceiver. The transceiver may be implemented as acomputer chip and antenna in a portable electronic device like asmartphone. Preferably, the transceiver of the host computer 104transmits the data as radio frequency signals over a wireless WIFI orcellular telephone network of the telecommunications system and network14 to the remote host computer server 105. The transceiver at the server105 may be configured to be part of wireless and/or land line in type.The communications system at the host computer 104 and remote hostcomputer 106 server may include compression and encryption capabilities.The data received is operated upon when it reaches the server 106.Server 106 will typically be a personal computer, workstation computer,or a rack mounted computer server. The server hardware is of a type likethat is used to process data/information like that described in U.S.Patent Application Publication 2009/0196493 A1, dated Aug. 6, 2009, andas described as part of the Cognitive Assistant that Learns andOrganizes (CALO) system developed by the AI Center, SRI International,of Menlo Park, Calif. Incoming data is parsed by the server depending onits type. The type of data is preferably meta-tagged at the sending hostcomputer end to facilitate parsing and processing by the remote hostcomputer server. The general types of requests include: 1) data loggingrequests; 2) command and control (C2) requests; and 3) query requests.

As previously discussed, near real-time data logging requests includethe storage and logging of data recorded by the internal and externalsensors. The data is received and processed for filing andstorage/logging in the memory segment folders database of the server.Examples of the types of sensor data and information received forprocessing and data storage includes storage of panoramic video, brainpattern activity imagery, sub-vocal signatures, and so forth and so on.Examples of Command Control (C2) or Command, Control, Computers, andCommunication (C4) information received includes information searchrequests and database build requests. Preferably, at least some portionthe correlation module 111 is located in computer 104 borne by the user,but optionally, correlation processing may be carried out completely onthe remote computer server 106. Preferably the portion of thecorrelation module kept on the host computer is relational databaseinformation commonly accessed, such as portions of the NCC database orfrequently accessed historical database data and information. In thepresent example C2 or C4 requests sent from the host computer to theremote server are transmitted to the AI and AI-like (i.e. PersonalAssistant that Learns SRI International) portion of the remote server.Examples of C2 or C4 requests include requests to stop and start theremote server; place the server on standby; or only perform logging whencertain internal or external parameters/thresholds are met. Forinstance, when AMR brain activity indicates the user's heart beat hasreached a certain level; AMR data shows heighted activity bypredetermined neurons indicating a “cat” is being observed in theexternal environment and an allergic reaction is starting to take placein the user's body; or when analysis of the video being logged inindicates a certain subject or activity is in the external surroundingenvironment. The interactive command and control module 107 of hostcomputer system 104 is operated to establish rules for controlling thesystem and rules on when, where, and how queries will be acted upon inmodule 109 and 111. Rules are typically established by the user 101 ofthe system 104. For instance, the user may use a menu to review dataprovided in response to a query prior to having said data automaticallyacted upon by a sub-system (i.e. a medication automatically beinginjected into the user).

Query requests sent from the host computer to the remote server 106 willbe transmitted to the AI and AI-like (i.e. Auto-Associative NeuralNetwork) portion of the remote server 105. Examples of query requestsinclude asking the auto-associative neural network to replay video ofthe last time you saw your Mom and Dad together during your life, or avideo recalling an appointment where the physician told you what typeand the medicine to take when your allergies to a “cat” reaches acertain pathology. The server then searches the memory segment foldersfor the information requested in the query. The information in responseto the query is then transmitted to communication portion of the remotecomputer and transmitted over the telecommunications system and networkto the transceiver of the host computer. The downloaded information isthen processed in the host computer for input into the user via userinput devices (i.e. head-mounted display HMD 135 systems or EMD 137electronic contact display lenses). The queries may be consciouslydirected by the user or automatically directed based on previously inputcommands. As mentioned earlier, rule sets which include thresholds thatdefine the numerical strength and relevancy of the data will typicallybe operated upon to determine whether located information found in thequery is transmitted as feedback to the user 101 and how it is actedupon once it is received by the portable host computer 104. It isanticipated that certain critical and/or continuously used informationwill be cached in memory stored on the portable system 104 RAM orhardware and that rarely used information will be stored in memorystored on system 106.

In contrast to the system described in FIG. 34 in which a user's hostcomputer interacts with a remote server on a telecommunications network,FIG. 35 describes a telecommunication system for interaction betweenpeople or machines according to the present invention. FIG. 35 providesa diagrammatic drawing illustrating the portable voicelesstelecommunication system and method between multi-participant humansand/or machines consistent with the present invention 100. For example,in FIG. 35 both a sending user 101 and recipient user 102 are operatingthe present invention 100 which includes a telecommunication system andnetwork 105, remote computer system 106, headgear 73 with internal andexternal sensor recording units and presentation units in headgear 73,and portable host computer system 104. Thus, the user 101 is able toremotely observe the environment that the Receiver 102 is in. In thisinstance the recipient 102 is taking care of the cat of the user 101.While observing Receiver 102, user 101 thinks of something he or shewants to communicate to the user 102 by operation of system 100. In thisinstance the user 101 thinks “Feed that cat.” Activated internal andexternal sensor units headgear 73 to include head-and-eye tracking andglobal positioning 424, brain activity 421, voice recognition orsub-vocalization 423 and video camera 168 readout respective informationto the host computer 104. Internal data such as AMR brain pattern andactivity data 257, sub-vocal recognition data 203 a, and external datafrom the panoramic video data comprising panoramic imagery and spatialaudio are processed into digital signatures that are translated intobinary computer code 168 a and 168 b respectively, that is graphicallydepicted by boxes with ones and zeros. The internal and external sensingmonitoring and logging module 109 represents pre-processed raw data andinformation read out from fast fusion sensor system processors locatedin the headgear 73 of the user 101.

As indicated by the bracket 425 a the data from the sensors units in theheadgear 73 is then transmitted to host computer 104 a. The input dataand information from the sensor units is logged into the file folders ofthe correlation module 111. As depicted in FIGS. 28-30 data is thenprocessed in the computer 104 and/or a remote computer 106 server toidentify CP's and NCC's between the internal and external data andinformational representations. The data and information is then storedin file folders in computer 104 and/or 106 memory. The neural networkincludes metadata comprising links back to the pre-processed sensor datawhich is filed such that it may be called up or later analysis orreplay. Unwanted data and information is disregarded in the process.Alternatively, and/or additionally, raw data from the sensors may belogged into a file prior to pre-processing for storage and potentialretrieval and pre-processing by the computer 104 a. As depicted in FIG.30, the neural correlations are then translated into machine or spokenlanguage that is transmitted from one sending user 101 (depicted as aSender) to another receiving user 102 (depicted as a Receiver).Filtering out noise, focusing in on specific neurons in the brain,focusing in on specific image patterns using video target tracking andimage recognition programs, strength of signatures, and performingcomparisons of subject data with historical data in the memory segmentfolders database are the processing techniques used by the system 100 todetermine “hits” and “matches” to determine neural correlates anddetermine the meaning of the data at a given time or over a given timeperiod are functions of the applications firmware and/or software incomputer 104 and/or 106. In the present example, neural activity in thebrain and actions in the surrounding video scene confirm that Receiver102 named “Bud” who the user and Sender 101 is “messaging (MSG)” with isto “Feed the cat.” The message constructed by the host computer 104 afrom sending user 101 to be communicated to host computer 104 b torecipient user 102 is derived by the host computer 104 a operating toidentify and construct the CP, NCC, and composing a correspondingmessage based on the CP's NCC's and relationships drawn and derivedtherefrom for transmission from the sending user 101 to the recipientuser 102 as illustrated in FIGS. 30 and 31.

The message is transmitted from computer output module 71 that includesa wireless transceiver module 58 a worn by or implanted in the user 101to the recipient user 102 over the telecommunication system and network.Both audio output 74 and video 72 graphic imagery for overlay onto theEMD are output for transmission in the present example. The hostcomputer 104 b transceiver worn by or implanted in the remote user 102receives any verbal or written information in a language he or sheunderstands because the communicated language has been correlated asdescribed in FIGS. 28-30 into a language understandable to the recipient102. And in a similar manner as described in FIG. 31 the translationtable 259 allows different CP's and NCC's to be translated andcommunicated back to another user over the telecommunication system. Itshould be noted that a translation table or key 259 may be constructedallowing two machines or a human and a machine to communicate with oneanother.

As depicted by the bracket 425 b, text, graphics, imagery, and audio maybe transmitted from transceiver 58 a between a first user 101 operatinghost computer 104 a and second user 102 operating host computer 104 b isdisplayed on each of the user's respective presentation devices. In thisinstance the input device for the recipient 102 is a head-mountedaudio-visual system like that shown in FIG. 19. The host computer 104 bwireless cellular capable transceiver 58 b receives audio signals 74 andvideo graphics 72 from the sender 101. Audio signals 74 received by theear bud speakers 138 a-b and video graphics 72 received on the EMD's 137a-b are presented to the receiving user 102 on his or her head-mounteddisplay. For instance, in this example, an augmented realityapplication, in which an arrow points to the cat that needs to be fed,may comprise the video graphics displayed on the Receiver's 102electronic contact lenses. Voice synthesized audio is received in theuser 102 ear buds stating “Bud, feed the cat,” is derived from sensorfusion of sensor analysis of user 101 sub vocalization, brain activity,and video imagery, which coincide and confirm the CP and NCC of the samethought “Bud, feed the cat,” which is announced in the user 102 earbuds. It should be noted that various types of human and machinelanguages may be transmitted in various computer and media formats tofacilitate communication between beings, machines, or a combinationthereof.

FIG. 36a illustrates the process of recording, processing, transmissionin module 109, and then FIG. 36b illustrates the subsequent relatedprocess of correlation and translation processing in module 111 of thesame internal and external sensor data according to the presentinvention 100. Panoramic audio-visual coverage is provided in certainembodiments of the present invention in order to record accuratedescriptions of what place cells in the brain perceive with respect tothe “Time”, “Place”, and “Spatial Resolution” in which the user islocated in the surrounding environment 160, and can provide a moredetailed recording than what the mind can remember. This is particularlyimportant in building realistic simulations that may be played back inwhich any portion of the scene may need to be created to achieve a senseof historical presence. The dotted lines 45 illustrate the FOV of eachof the lenses of camera 2 a-d. The dashed lines 162 illustrate the FOVof the left and right eye view of the user 101. The cat illustrates realworld subject which is the CP 161 forward of the eyes of the user. Theoval shape indicates the brain 167 of the user and the items within theoval specific neurons 164 firing in the brain 167 which is the CP whichdefine the NCC 159 related to the same subject 161 in the surroundingenvironment outside the head of the user. In FIG. 36a the panoramicvideo camera system records images 114 a-d using associated videocapture device(s) like those shown in FIGS. 2a-d , 3, 4, 5 a-b, 10 a-c,11. Previous optics described in this specification and in thebackground of the invention site various panoramic capture devices of atype and design that may be used in the present invention. The imageshave adjacent or overlapping FOV coverage such that a spherical FOVscene is recorded. Alternatively, overlapping imagery may be captured inorder sample out stereo graphic imagery. FIG. 36b is a depiction of thefour resulting images 114 a-d. The camera images 114 a-d are shown ascircles to illustrate hemispherical images recorded with fisheye lenses.The rectangular shapes indicate a frame. In the present example apierced ear camera facing outward from the ears of the user 101 torecord adjacent images c and d 256 a to the front and back of the user'shead, and cameras a and b 256 b record adjacent images to each side ofthe user's head. As indicated at the bottom of FIG. 36b , the images 256a-d are then optically manipulated and/or digitally processed by acomputer and output so that at least some portion of the imagery issampled for additional processing or display for viewing by a local orremote user. In the present example the CP 161 the user is focused uponis sampled from images 256 a and 256 b to provide for display by thecomputer 104. The images are presented to the user on the EMDs 137 a-bas undistorted images 246 a-b.

FIGS. 37a-c are schematic diagrams illustrating system 100 dynamicinteractive multi-region-of-interest (ROI) processing and display for alocal or remote user. FIG. 37a through FIG. 37c correspond to the samedata and information described in FIG. 36a and FIG. 36b .Region-of-interest image capture sensors of a type that may beincorporated in the present invention are incorporated by reference andwhere cited up front this patent application. FIG. 37a illustratesimages 114 a-d that are recorded by a panoramic camera system with fouradjacent side by side panoramic lenses facing outward about a point at90-degree intervals. The FOV of the images may be adjacent formonoscopic, bi-ocular, or overlapping imagery for stereoscopic imaging.In the present example the imagery is stereoscopic presenting at leasttwo views of all surrounding subjects in the surrounding environment160, as indicated by the same subjects being imaged by multiple fisheyelenses. Dashed lines in FIG. 37a indicate the ROI 272 that the ROI imagesensor is focused upon based on the user's selection. FIG. 37billustrates a resulting composite undistorted spherical FOV image frame245 processed for viewing by the user in which any portion of thespherical FOV panoramic scene shown in FIG. 37a may be panned and zoomedupon by using interactive input devices. The x-y-z Cartesian coordinates275 are shown to illustrate the panning capability. The host computerdoes this by sensing the viewer's head and eye position, relaying thosecoordinates to the ROI image sensor, and sampling out the ROI the useris looking at in the recorded spherical FOV. Images may be sampled outfor monoscopic, bi-ocular, or stereoscopic viewing depending on thecamera, processing, and display system. FIG. 37c illustrates a resultingcomposite undistorted spherical FOV image frame 245 processed forviewing by the user in which three images are sampled out of the imagesshown in FIG. 36a and FIG. 37a for display. A first enlargedsub-windowed ROI image 274 is the receiving user 102 at the remote end,a second enlarged window ROI image 274 of the sending user 101 at thelocal end, and finally the background image is a scene of a cat 273 infront of the other user 102 which both users 101 and 102 are looking atand discussing in their two-way video teleconference. In this examplethe cat 273 is the image observed by the local and remote user of thesystem related to the cat CP 161 in the user's mind and the subject cat159 observed in the surrounding environment 160 as denoted in the FIG.37b . As those skilled in the art will realize, multiple ROI images andgraphics across the spherical panoramic images may or may not beselected and windowed over a local or remote background depending onlocal user and remote user preferences.

FIGS. 38-49 illustrate embodiments and applications of the inventionthat may be derived from the Life-Logging and Memory EnhancementAssistant (LLMEA) 100 system. FIG. 38 is an illustration of a screenshot of the graphic user interface (GUI) 64 of on a host digitalcommunication device, also referred to as a portable host computersystem 104, like a smartphone 151, acted upon by the user 101 tointeractively control the operation of the present invention 100. Thesmartphone includes all the standard smartphone functionality, hardware,firmware, memory, microphone 51, audio speaker, camera 279, and soforth. But, additionally, as described throughout this specification,the smartphone 151 includes specific applications which include hardwareand firmware that interact and operate with other systems, components,and applications unique to the Life-Logging and Memory EnhancementAssistant (LLMEA) 100 system that comprises the present invention. Forinstance, the graphic user interface 278 includes a touch screen 276menu entitled “Selection Menu” 65 that a user 101 manually operates toselect device parameters that control and command the user interactivecommand and control module 107, internal and external sensing monitoringand logging module 109, and correlation module 107. Once the user setsthe parameters the device may automatically act upon the settings of theuser whenever the device 151 is tuned on. The options are listed on thetouch screen of the smartphone such that the user interactively selectsthem. When the user touches a topic and it changes from red to green theapplication is activated. By touching the application again the topicstext turns red and the application is turned off. Multiple relatedfunctions may be selected to enable or disable applications that runsimultaneously. The user has the option to select options which makeoperating the device in a hands-free manner, and the user has theability to lock the screen so the device or selections are inadvertentlyactivated. In some applications like “Query” or “Named Percept” adatabase of features or correlates are searched upon. Commands areentered into the device 151 to control and drive biometric and brainactivity sensing systems, surround sensing systems, the correlationsystem, smartphone operation to include interactive feedback systems,and the telecommunication system. Text, touch, or voice commands willtypically be entered into the GUI 278 of the host computer 151 in orderto activate an application to function. Similarly, buttons may bepressed to turn on and off the host computer, to include biometricidentification and security systems. As illustrated at the bottom of theGUI touch screen, the GUI also includes “Zoom” and “Pan” 277functionality. The user may press the arrows on the touch sensitivescreen of the device 151 to zoom and pan the spherical FOV scene, andpresses the center button to zoom in and out on an item. Alternatively,the operation of the user interface be done hands free using the voicerecognition system, sub-vocal recognition system, hand or facialgestures, ROI automatic windowing as shown in FIG. 37b-c , or the brainactivity sensor system. The system may answer back to the user bydisplaying responses via text, imagery, user force feedback glove orbody suite, image display, conventional audio speaker or earbud, orvoice-synthesis audio speaker that may comprise part of and are withinthe scope of the present invention 100.

FIGS. 39a-39b each illustrate a Graphic User 101 and/or 102 Interface(GUI) 278 menu screenshot 189 for commanding and interacting over asocial media network embodiment of the present invention referred toherein as “MindShare.” A picture of the user 101, here referred to as“JC” and/or “John Curtis” is provided in the top right-hand corner ofthe GUI. The GUI may be hosted on the display of a host computer system104, like a smartphone 151, HMD 135, or EMD 137, or other portablecomputer display borne by the user. Interaction with the GUI may beaccomplished by the user operating various input devices such as thetouch screen of the Smartphone, or using voice recognition,sub-vocalization, brain activity, eye gaze, or a combination thereofwith the EMD and HMD. Alternatively, other user(s) 103 not wearingsystem, like user 101 and 102, may receive data derived from system 104and/or 106 using devices, such as a “VideoRoom” (see Pub. No.20100045773 and Pat. Nos. 5,495,576, 5,130,794, 4,656,506). MindSharerefers to a life experience sharing site (see FIG. 39a ). The LLMEAsystem 100 that comprises the present invention is operated aspreviously described in this specification to record, process, log, andpresent internal and external data and information derived by saidsystem over the telecommunication system and network 105. Remotecomputer servers 106 may operate in a cloud computing arrangement on theInternet to process and store data and information derived from saidportable system 104. The network and associated sites may be hosted onthe public domain, like the Internet over the Global Information Grid(GIG), or on a closed domain and network. If the Internet is used it maybe part of the telecommunication system and network 105.

In FIG. 39a-39e a user of the network uses the menu to select a livefeed or previously recorded experience. The selection cursor isillustrated by an arrow. The user of the system employs the menuselections to define system parameters of system 100 and to allowvarious accesses to other users. For instance, the user may selectvarious levels of accesses for himself, his friends, and for unknownvisitors. Preferably, the owning user is the administrator and hasadministrative privileges. Also preferably, the user sets very rigidpassword and security features to protect his or her privacy and guardagainst identity theft. For example, in FIG. 39a the user may activatevideo logging by selecting the “On” button shown at the middle left ofthe menu under “Primary Selections”. A window on the menu allows theuser to select what internal and external sensors are turned on forlogging and what is being released to other persons or machines. Thetype of video logging, be it monoscopic, binocular, stereo, or sphericalFOV coverage may also be chosen along with other selections under“Presentation Type”. The user may also activate information derived fromthe brain activity sensing system, or for that matter, any of theinternal or external sensing systems that comprise system 100. Thus, awindow on the menu to select what the user is thinking and sensing inhis or her brain may be logged by system 100. The information queriedmay comprise a live feed or prerecorded feed. The user may operate aslider bar on the menu to search data and information logged over time.Here indicated by 2011, 2010, and so on.

Typically, the user operates the menu 278, shown as a screenshot 189, tochoose what content to release to other users on the social network.Various permissions may be given to other users using menu selections.The user may even give machines permissions for control. The user mayuse the menu to log into a simulation or to control a robot. Forinstance, a user may choose to transmit a live feed of what he issensing and what he is focused upon in a given environment over thesocial network. Thresholds and rules may be set up in the system 100 tofilter out information the user does not want released over the network.A window with the user's face, name, contact information, and so on maybe placed on the menu to identify the user. A window of the live contentthe user is logging and transmitting may be placed on the menu. Forinstance, as illustrated in the present example, “jc” the user 101 isviewing a cat in the user's FOV. The user “jc” is wearing left and rightelectronic eye mounted display EMDs 137 a-b contact lenses withintegrated cameras which are transmitting his left and right eye imagesto computer 104 and each EMD 137 a-b. Additionally, the brain activityof the user is being logged by computer 104. Computer 104 is concealedby a skull cap and wig that the user is wearing. In the present examplethe images the user is viewing are being processed and displayed on themenu along with an Augmented Reality overlay of text identifying whatthe user is viewing. The Neural Correlates of Consciousness (NCC)identify what the user is viewing. The NCC is determined in the computer104 by performing processing operations on the sensory informationcoming in from the live sensor feeds and correlating it withmulti-sensory historical data that has been previously logged in by thesystem 100. In this manner and as previously described identification ofthe cat may be displayed as the textual augmented reality overlay “MyCat” on the see-through electronic contact lenses the user is wearing asshown in FIG. 39 a.

FIG. 39b is a graphic representation of a menu 278 shown as a screenshot189 an administrator, here the user of the system 100, selects tocommand and control what internal and external sensor systems record,process, log, and present. Selected items chosen by the user areindicated within circles that may be selected as a menu control optionas part of the GUI of computer 104. In FIG. 39b the user has chosen torecord and send a live feed of what he is thinking in his frontal lobe,what he is hearing in both ears and seeing in both eyes. He is lookingat a cat as indicated in the two boxes showing his FOV in the upperright side of the menu that have the textual overlay “My Cat” next tothe cat he is observing as his CP 161-163 in the local surroundingenvironment 160.

FIG. 39c is a graphic representation of a menu 278 shown as a screenshot189 the system 100 administrator, here the user, operates to makeselections to share his or others previous experiences logged into hissocial network site. The site may be hosted on a computer the user ownsor on a server operated by an Internet provider. Examples of selectionsinclude “My Favorite, Most Viewed, Top Rated, and Most Recent”. Thecurrently active portions of the menu are indicated in solid bold lines,the not chosen and inactive portions are shown in un-bolded lines.Additionally, a “Recent History” status window may be included to showwho is visiting or has visited the site and their activity on the site.An oval with dashed lines illustrates a user 101 who is currentlytransmitting a previously recorded video feed of their high schoolgraduation to a user 102. A payment system may be integrated into theMindShare embodiment of system 100 that charges users to download liveor pre-recorded MindShare information.

FIG. 39d is a graphic representation of a menu 278, shown as ascreenshot 189 of system 100. An administrator, here the user 101, makesselections to conduct a live video teleconference with other users 102logged into his social network site. Active selections are shown inemboldened lines and text indicating who is participating in the two-wayor multi-point teleconference, and that video logging and memoryenhancement sharing is turned on or active. Inactive systems are shownin un-emboldened lines. If certain settings are activated andpermissions granted the thoughts in the mind of user involved in theteleconference may be queried while operating the system 100.

FIG. 39e is a graphic representation shown as a screenshot 189 of a menu278 the user 101 operates to make selections to conduct a search onlogged information derived from his use of system 100. As shown in theupper right of the menu, the user 101 has initiated a search. The useroperates the present invention to conduct a search of data orinformation by either using interactive input devices which allow him orher to type, touch, think, or speak into input devices, such as with akey board, touch sensitive, brain activity, voice recognition, or asub-vocalization system of the present invention. As indicated in theupper right of the menu in a windowed rectangular box, the user hasindicated to system 100 that he or she wants to conduct a search usingthe keywords “11 Nov. 2011 Church”. The user typically activates thesearch engine areas listed as time, place, subject, or activity. Asindicated at the middle left of the menu search thresholds can be set.Thresholds establish the strength of a response to a query ofinformation. The higher the threshold, the stronger the correlation orlikelihood that the information presented matches the search criteria.As indicated at the lower left of the menu window other persons, besidethe owner, may also be queried. The persons whose experiences (i.e. viapanoramic video and brain activity) are logged may be deceased orliving. Additionally, unique areas such as immersive interactivesimulations like that described in FIGS. 41-42 and robotic control ofuser robots like that described in FIGS. 41, 43-45 a-d may be activatedin the “Other” selection window located at the lower right bottom of themenu of the search engine may be selected. U.S. Patent ApplicationPublication No. 2009/0156955 by Jung et al., dated 18 Jun. 2009,entitled “Methods and Systems for Comparing Media Content” presentssystems and methods that may be integrated with the present inventionand is hereby incorporated by reference.

FIG. 40a-b shows another embodiment of a menu screen which comprises asimple to use homepage for social interaction of users of the presentinvention 100. The homepage comprises conformal imagery icons 281 in theform of computer graphics and imagery like that available in the form ofa Google Earth™ background overlaid with the bodily representation ofthe user participants of a teleconference. The participants operatesystem 100 to interact on the online network. FIG. 40a-b is a graphicrepresentation shown as a screenshot 189 of a menu 278 of theinteractive display screen 64 the user 101 operates to make selectionsto conduct the teleconference. Computer 104 borne by the user preferablyoperates on the image recorded by the external video camera sensorsystem, including peripheral video camera sensors, to clip images of theuser's body without the background and overlays the interactiveconformal images over the geographic information or in an appropriatespace. An overlaid silhouette or a photograph of the users may beselected using an input device that is integrated with computer 104 toinitiate the video teleconference. But once the video teleconferencestarts, the clipped video silhouettes preferably change into a livevideo representation of the subject for the duration of the videoteleconference. A portable wireless computerized video clipping systemfor augmented reality overlay onto a graphic or video background of atype consistent with the present invention that may be incorporated intothe present invention was presented at the International Society forMixed Media and Augmented Reality Symposium 2001 by Hirokazu Kato et al.of Hiroshima City University, Japan, kato@sys.im.hiroshima-cu.ac.jp.,entitled Block Assembly in Augmented Reality [29]. FIG. 40b is a graphicrepresentation of the interactive immersive teleconferencing systemaccording to FIG. 40a in which one of the users in the teleconferencehas been enlarged to fill the user's display. As previously describewith reference to FIG. 36 a-b and FIG. 37 a-c various ROI windowingfunctionality may be implemented within the present teleconferencingsystem social interaction homepage/website. Finally, as indicated inFIGS. 40a and 40b a local user or remote user with permissions mayinteractively select the “MyView” intuitive text or icon overlaid on thedisplay to receive a live video feed 282 of what he, the user 101, isobserving or alternatively receive a live video feed 282 that a remoteuser 102 is observing. If the system 100 includes panoramic video camerafunctionality, and the functionality is activated and authorized, theusers of the teleconference may select panoramic video feed to zoom andpan upon so they can experience the surrounding environment at their oranother user's remote location. Typically, display devices users mayoperate to observe the transmitted video during the video teleconferenceinclude a smartphone, HMD, or EMD system like those previously describedin the present invention. Users logged into the “MyView” portion of thesystem that use a spherical panoramic camera system like that disclosedin the present invention are able to view the user's face and his or hersurrounding scene. Still further, as indicated in FIGS. 40a and 40b aremote user with permissions may select the “MindShare” text or iconfrom the menu to receive a video feed of what the user is thinking andfeeling to experience the user's surrounding environment in a similarimmersive manner via sensory feedback mechanisms mounted on the user'sbody (i.e. mind stimulation feedback systems and body worn forcefeedback systems). Correlation and translation systems that enable“MyView and “MindShare” functionality are described in devices disclosedin FIGS. 1-24 and methods and processes disclosed in FIGS. 25-35 and thesupporting text. An online payment system may be integrated into the“MindShare” or “MyView” system to charge people to download information.In this manner people, such as celebrates, could charge fans forexperiencing their lives vicariously as a prerecorded or as a live feed.

FIG. 41 is a block diagram describing systems and methods for modelingTask A 284 a and/or Task B 284 b, for simulation, stimulation, and/oremulation in Option A 285 a, B 285 b, and/or C 285 c to which data andinformation derived in Steps 1 and 2 283 during operation of theportable data logging and memory enhancement system 100 may be applied.Step 1 is recording and Step 2 is storing the computer memory historicaldatabases derived from internal/external logging system for use inperforming Task A or Task B.

Task A 284 a is to incorporate internal data and information derivedfrom the logging system 100 for construction of a computer-generatedsynthetic model of the body and cognition of the user 101. Task A 284 ais the action of computer modeling the user 101 based upon data andinformation that comprises the relational database 287 derived duringthe logging process 283. Correlation of the data and information may beperformed at any point after recording the data and information. Forinstance, computer data from an eye tracking unit a user 101 is wearingindicates the user is looking in a certain direction where the subjectis a “cat”, and where the brain has historically and repeatedlycorrelated the image of a “cat” with a certain minimum brain activity.In this manner, this minimum required relationship between the outsideworld and the brain defines the neural correlates of consciousness 166necessary to recognize the conscious percept 159 of the “cat”. Thepreceding correlation can be drawn upon at any time thereafter (i.e.Time 2) to define whether or not the consciousness of the user 101 isfocused on thinking about a cat, either by system 100 analysis at Time 2of just brain activity or external stimulation impacting on the sensorysystem of the user. Additionally, the likelihood of the person thinkingabout the cat may be calculated in a certain environment. In otherwords, to calculate the likelihood of what the user is thinking, dataand information from both internal brain activity and physiology, or theexternal surrounding environment, may be analyzed using the system 100.Correlation calculations may be done at the Time 1, or data may beperformed at a Time 2, depending on the design of the system 100. Andinitial correlations may also be correlated with later brain to brain,brain to environment, and environment to environment correlations toderive what a user is thinking about. In other words, the system 100 mayperform correlations to other correlations, and correlations correlatedwith other pieces of data and information, to define what the user isthinking, and to drive the system 100 query, process, or action.Statistical computations performed by the correlation module 111 of thesystem 100 on logged data and information are operated upon by thecomputer to modify, strengthen, and/or reduce the body of evidence inthe computer that identifies neural correlates of consciousness andconscious percepts of the user 101 or 102.

In Option A 285 a is a user 101 in the real world wears an interactiveinput/output device and takes the form of an avatar in the simulation tointeract within a computer simulated environment 190. In Option B 285 bthe user 101 is modeled with historical data and AI is placed in acomputer simulated environment and various scenarios are tested. Andfinally, in Option C 285 c logged data and information derived fromsystem 100 that reflects the user 101 is loaded into a robot 286.Actions by users, subjects, and objects modeled in the syntheticenvironment may be constructed using computer image texture mappingthree-dimensional modeling software or firmware. In such an instance,historical records of responses to certain stimuli of a user or objectin the surrounding environment are captured by the video logging system.For instance, brain activity captured by the portable AMR system born bythe user is associated with an image of a physical action in surroundingenvironment, such as an impending automobile collision. Then when asituation in the synthetic environment is encountered the object or userreacts in a similar manner. Or in the simulation, alternative courses ofaction may be tested. Lessons learned in the simulation may be appliedto real life situations in order to improve the performance of the useror others. Besides an actual living user wearing interactive devices tointeract as an avatar within the synthetic environment, historical datafrom the video logging system may be introduced into a machine to drivethe simulation. In this instance the machine operates to take the formdrives the simulation. Avatars may represent another being, machine, orthe user. Avatars similar to the user may be constructed in thesimulation using simulation modeling techniques and tools know topersons skilled in the art of live and virtual computer simulations.

Task B 284 b is to incorporate internal data and information derivedfrom the logging system 100 for construction of a computer-generatedsynthetic model of the environment 284 b surrounding the user 101. Asdisclosed in patent '794 and '576 by the present inventor, syntheticenvironments logged using the present invention are built by mappingimagery and deriving shape data recorded by a stereoscopic and/orpanoramic camera system facing inward and outward that is worn orcarried by the user onto three-dimensional wireframes constructed usingcomputer graphics software and methods. The synthetic environment isalso constructed by mapping audio recorded by a panoramic camera systemfacing inward and outward that is worn or carried by the user and thatcan be associated with subjects and objects constructed using positionalaudio software and methods. In both cases, the imagery and audio may bedriven by associated physics and actions of subjects using software andmethods applied to the constructed models. Interactive devices such asdata gloves, data suites, head-mounted position sensing systems,geospatial positioning systems, eye-tracking system, joysticks,trackballs, mice, keyboards, as well as other devices, may be used tointeract with the simulated environment. Interpolation and extrapolationin the construction of objects and subjects not completely heard orobserved can be constructed based on rule sets to complete the back sideof scenes and expand upon portions of the synthetic simulatedenvironment. As described in Option A, B, and C of FIG. 46, syntheticbeings are constructed by incorporating logged data using the actionsand information recorded and derived using the method and system 100 inthe present invention, which will be discussed below in more detail inthe next few paragraphs.

FIG. 42 is a block diagram of the present invention 100 integrated witha simulation system 190. Stored data and information 287 derived fromthe LLMEA 100 system is coded into computer data and information that iscompatible with the simulation system. Three-dimensional objects,beings, and their actions are constructed and provide an input source284 from the logging and memory enhancement system that comprises thepresent invention. For example, the host computer 104 that controls thebrain activity sensing system, surround sensing system, and acorrelation system to output data and information to construct thesimulation database 287 which the to the simulation system 190 operatesupon.

The simulation system 190 may operate on data being transmitted to thesimulation from the logging system 100 in a live near-real time manner.As disclosed in Option A 285 a of FIGS. 41 and 42 a live user mayinteract in near real time with simulator interactively by controllingan avatar. The user may receive interactive feedback transmitted overthe telecommunications network to the user borne portable feedbacksystems, like a HMD, that comprise a component of the system.

Alternatively, the simulation system 190 may operate on historical dataand information from system 100 that has been pre-recorded and stored inthe memory of the simulation as disclosed in Option B 285 b of FIGS. 41and 42. In which case the avatar that represents the user is not livebut is modeled using historical data and information derived fromoperation of the logging system 100. The example simulation system maybe that like the panoramic video based virtual reality simulationsystems described in U.S. Pat. No. 5,495,576 by the present inventorthat illustrates the potential use of the inputs derived from the datalogging system. Audio, image, shape data and information of thesurrounding environment and user derived from system 100 is input 284 ainto simulation 190. User 101 internal and cognitive and physiologicaldata and information 287 derived from system 100 is input 284 b intosimulation 190. Because of the volume of data and information comprisingthe cognitive and physiological database 287 representing the user acomputer memory storage unit 261 is required. The user mass storagedatabase 261 interacts with the artificial intelligence or artificialintelligence-like simulation module 285 b to mimic the actions of theuser 101. The data and information from the brain activity sensingsystem of the present system 100 are input into the simulation system todrive the A.I. processor 285 b that mimics user. The A.I. portion of thesimulation system 285 b may closely mimic an avatar that represents theuser of the logging and memory enhancement method and system 100 thatcomprises the present invention. The computer system 104 and 105 mayprovide direct input into the simulation. Correlation systems like thatdescribed as U.S. Pat 2009/0196493, dated 6 Aug. 2009, by Widrow et al.entitled Cognitive Method and Auto-Associative Neural Network BasedSearch Engine for Computer and Network Located Images and Photographs),CALO, and SIRI system that communicates may be incorporated into the A.Isystem to provide feedback output to the user of the simulation.Preferably, the simulation comprises a visual simulator, to provide auser visual and audible feedback. But alternatively, the simulation mayalternatively comprise a non-visual simulation system. Using these toolsand methods the simulation the A.I. portion of the simulation system 285b may closely mimic an avatar that represents the user of the loggingand memory enhancement method and system 100 that comprises the presentinvention. A physical interactions processing model 266 operates toprocess the user actions in the computer-generated world. All entitiesin the computer-generated world are managed of in the visual simulation267. In this instance the modeled being's actions are based upon storeddata and information that models the surrounding environment 284 bpreviously derived from the logging system and input into thesimulation. Computer mass storage 268 of entity information coded intothe memory of the simulation 190, to include three dimensional models ofentities with texture mapped imagery logged by system 100 that comprisesthe world that surrounds the user, is provided and is called up by thevisual simulation 267 based on time and place commanded by the user ofthe simulation. Three-dimensional shape information may be constructedmanually or autonomously based on lighting and shading of video imageryrecorded by the surround sensing video camera system. Surround videocamera imagery may be texture mapped onto associated three-dimensionalmodels constructed from an integrated surround Laser Radar (LADAR)system. A user like a being or machine, like that illustrated in FIG.45, provides an example of a portable integrated surround video andRADAR system according to the present invention. Information and dataderived from the logging system 100 in the present invention are inputinto the computer simulation system as panoramic imagery, spatial audio,shape information, and user physiological data and information. Thelogging system may be tailored for collecting data specific to the typeof simulator the logged data will be used in. For instance if astereoscopic, auto-stereoscopic, or holographic display is to beconstructed using the logging system 100 then at least two views ofevery scene and object will need to be logged so that a correspondingvisual model may be built and displayed. Similarly, if anambisonic/surround sound system is to be constructed then a plurality ofmicrophones facing outward from the subject will need to be logged sothat a corresponding audio environment can be constructed later in thesimulator.

It should be noted that logged data and derived information according tothe present invention may be operated upon by the simulation 190 todrive and influence decision making of users and robots in the realworld. For example, an actual user may transmit his location into asimulation where the user is represented as an avatar. Given the user'ssituation in the real world, the simulation runs fast forward todetermine likely outcomes. Then the simulation transmits back to theuser or robot in the real world recommended course of action to take inthe real world which are likely to yield the best outcome for the user.As discussed earlier a user or recipient of the derived data andinformation 287 may share live or pre-recorded instances of theirexternal or internal sensory logged data or derived information withother users of a social network that has access to system 100 over asocial networking system to assist in making decisions based on theoutcome of the simulation.

Still alternatively, it will understood by those skilled in the art ofsimulations that the simulation 190 may operate on both live and storeddata and information derived from system 100. In this instance thesimulation accommodates both live input and output as described inOption A 285 a and virtual or constructive input and output as describedin Option B 285 b of FIGS. 41 and 42.

Finally, as shown in Option C 285 c of FIGS. 41 and 42 a simulated robot286 may be loaded with the user's individuals historical data andinformation to emulate the original user in the simulator independentlyfashion. Or additionally, it is known to those skilled in the art that auser may remotely pilot or interact with a real robot 286 in a liveinteractive fashion outside but in communication with the simulator.

FIG. 43 is an exterior perspective drawing of a robotic system 286 of atype that may be integrated with the present invention. The exterior ofthe robot incorporates external and internal sensing systems previouslydescribed in the system 100 of the present invention. The robotincorporates data and information derived from the logging portion ofthe system 100. The robotic system may include an A.I. or A.I. likesystem like CALO. The host computer system 104 (i.e. a smartphone) and106 (i.e. a remote computer server on a network) may provide directinput into the robotic system. The computer simulation system may belike that described in the Widrow et al. patent and the CALO systemwhich logs data in and derives “neural correlations of consciousness”from external and internal sensor outputs is incorporated to define theactions of the robot. Using these tools and methods the robot mayclosely mimic the user or users from which data was derived. The roboticsystem includes an electrical generation, storage, and control system,propulsion system and other components standard to a robotic system andwell known to people in the robotics industry who construct roboticsystems that attempt to emulate human beings.

FIG. 44 is a block diagram disclosing the general method of using theinformation derived from the data logging system 100 to drive theactions of a simulation system or robotic system 286. A robot thatmimics at least some portion of one or more individual personalities ofa user 101 is constructed. The robot may be constructed and sent into ahostile environment that only a specially constructed robot is able tosurvive. The hostile environment may be deep in the ocean or outerspace. Historical data and information from the video logging system maybe introduced into the software and firmware of the robot to drive theactions of the robot. The robot's computer may incorporate artificialintelligence or rule based computing to drive the robot. Still referringto FIG. 44, the data and information from the brain activity sensingsystem of the present system 100 are input into the onboard databasederived from the logging system memory of the robot system to drive therobot's actions. The composite host processing system of the hostcomputer system 104 or a remote computer processing system 106 mayprovide direct input to the robot. The robot may include a userinterface C3 module 107 equipped with CALO and SIRI like software andfirmware applications that run on the host computer that communicate toa being or another machine. Correlation systems like that described asU.S. Pat. 2009/0196493, dated 6 Aug. 2009, by Widrow et al. entitledCognitive Method and Auto-Associative Neural Network Based Search Enginefor Computer and Network Located Images and Photographs) are includedonboard the robot. Internal and external sensing monitoring and loggingmodule 109 processes incoming data from the robots surround video camerasystem and other sensor systems and transmits the data over system bus208 for processing by correlation module 111. The data and informationfrom the internal and external sensing monitoring and logging module 109and correlation module 111 is stored in the relational database storage287 that includes the user 101 derived NCC database 288. This data maybe queried by the robot 286 or a remote user 101-103 over atelecommunication network and system 105, or in much the same way NASAtoday interacts with deep space probes.

FIG. 45a-d are a series of illustrations showing the present inventionintegrated onto a robot 286 with a plurality of sensor array 290 thatincludes a visual system that comprises a camera system 2 that includesat least one image capture unit 114, an audio unit that includes atleast one microphone 51, and a three-dimensional digitizing systemcomprising at least one small conventional radar or LADAR unit thatincludes at least one radar or LADAR 42 unit. The camera 114, microphone51, and LADAR 42 each have overlapping field-of-regard (FOR) coverage292. The overlapping coverage enables each of the array of sensors torecord an image, audio, radar or LADAR signature of a given side of auser within the FOR coverage of the each array 290. The array isoriented outward from the robot. Images, audio, and shape signaturesfrom adjacent and/or overlapping arrays may be sampled and processed bysystem 100 to record any portion of the surrounding FOR coverage about auser. The array 290 may be placed on a rigid or flexible materialcovering that is worn by a user to cover an entire or only a portion ofthe user. The user may be a machine or being. An exoskeleton may beincorporated to help support the weight of the support material on whichthe sensor array and associated support electronics are situated. Theexoskeleton may be hidden beneath the array when worn by a user. Image,shape, and acoustical signatures from each of the arrays are transmittedby a plurality of cables 7 a-c (i.e. wire, fiber optics, wirelesstransmitter or transceiver (not shown)) to a signal processing means ofsystem 104 as illustrated in FIG. 44. FIG. 45a is an enlargedperspective drawing of the exterior of the robot with the sensor arraycalled out in a circular area in FIG. 45b for descriptive purposes. Aplurality of array 290 may be placed side by side in a continuousoutward facing manner forming the curved outer surface of the robot'shead in order to achieve substantially spherical composite coverage ofthe sensor arrays mounted on the robot. The sensors are securelyfastened to the robot's head by conventional means, such as adhesive ormounting screw 289 or adhesive (not shown). FIG. 45c is a perspectivedrawing of a single array 290. FIG. 45d is a diagonal sectional diagramDD of the sensor array shown in FIG. 45c . As illustrated in FIG. 44,signatures from the array are operated upon by the robot's host computerprocessing system 104 to guide the robot, and data and information isselectively filtered and stored into memory to build a database derivedfrom the correlation system. The host processing system 104 continuouslyinterrogates the correlation system 111 as new information comes intothe user interface Command and Control (C2) module 107 operates to drivethe action of the robot 286.

FIG. 46 is a block diagram illustrating the application of inputtingdata and information derived from the operating system 100 into livingbeings. This is accomplished by applying data and information derived inSteps 1 and 2 283 during operation of the portable data logging andmemory enhancement system 100 to Tasks A 284 a and/or B 284 b, Options D294 a and/or E 294 b. To this end the system 100 is operated to record,process, and store historical data and information, depicted in block283, of a user 101 at a given time one (T1) and location/place one (L1).Individual times and locations may be aggregated to record eventsequences for recording, processing, storage, analysis, and presentationwithin system 100. These sequences are referred to as T1 to the nth andL1 to the nth. The recorded data and information may undergo variouslevels of processing prior to being introduced or acted upon by computer104. Option D 294 a is a method in which historical data and informationrecorded and processed by system 100 is operated upon by at least oneelectronic device to stimulate physiological processes of a patientand/or recipient user. For example, to stimulate the brain of arecipient, to re-stimulate existing brain cells, stimulate the growth ofnew brain cells, and/or stimulate implanted brain cells. FIG. 46 alsodiscloses an embodiment of the present invention, referred here asOption E 294 b, which incorporate the derived data and informationlogged by the system 100 into computer systems and biological systemsthat drive and regulate the physiological state of a recipient user on alife support system. The internal and external logging module 109 andcomputer correlation module 111 of system 100 operate on the recordedand stored data to build a historic database. All data and informationsubmissions have at least a time stamp, which preferably includes thetime and date, and a location or global positioning system stamp. Eachinstance of a submission to the database starts at Time 1. The data andinformation are processed by system 100 to identify CPs and derive NCCs.Databases of logged data and information are typically stored incomputer 104 or 106 memory where the data and information has beentranslated into computer language which is able to be processed bycomputers 104 or 106. Computers 104 or 106 process new and old data andinformation to build a database that represents a body of knowledge andunderstanding based upon input external and internal sensory datacollected by modules 109 and 111. Data and information operated upon orpresented to a user at T2 may comprise raw data and information, dataand information derived from raw data and information, or metadataderived from system 100. The stored data and information may includeimagery, audio, and other sensory information along with various othertypical computer operating system and application code. The computer mayinclude A.I., A.I.-like, or no A.I. application functions. Preferablythe data and information are put into the computer is translated into acommon computer language for ease and speed of manipulation by thecomputer operating on the data and information. The computer system 100may have the capability to translate analog signals into digital signalsand vice versa depending on the exact design of the system and desiredapplication. For instance, recalled historical data and informationderived from the video logging system 100 may be replayed by use of acomputer video graphics card or processing module in system 100 in anaudio-visual multi-media format compatible with a user presentationdevice (i.e. a head-mounted display) to re-stimulate existing braincells, stimulate the growth of new brain cells, or stimulate implantedbrain cells. Modeling the body system of the user is derived from theinternal and external sensors that record and store the activity of theuser in a given environment by incorporating system 100. Within thesystem 100, modeling the body 284 system of the user may involvereplicating any one, a combination, or all of or some portion of thephysiology attributes that make up a user, to include the: nervous,muscular, circulatory, respiratory, gastrointestinal, integumentary,urinary, reproductive, immune, endocrine system. External sensors thatrecord the peripheral areas of the body may be used as input incollecting information that is operated on at a later time to helpreplicate a user. For instance, the camera sensor 2 c in FIG. 1a-c , isoperated at Time 1 to record facial features of the user. The facialrepresentation may be correlated with brain activity and imageryrepresenting the conscious percept the user was focused upon at a giventime and place. Those same attributes may be replicated when the user issimulated in the same or a similar simulated environment. In this mannera body of information is built up in a database that the computersimulation system may be operated upon to emulate the user.

Option D 294 a describes a method of stimulating the senses of a user101, 102, or 103 with corresponding historical data and informationderived from a user operating the internal and external logging system100. Electronic devices, including the computers 104 and 106, areoperated to playback portions of the data and information recorded bythe system 100 into electronic user feedback and presentation devices.Feedback and presentation devices of a type that may be operated toprovide feedback data and information to the user include: Panoramicdisplay systems for visual stimulation of the user; surround soundacoustical systems for auditory stimulation of the user; haptic andforce feedback systems for tactile stimulation of the user; smellreplication systems for olfactory stimulation of the user; and tastereplication systems for gestation stimulation. In order to increase theretention of data and information introduced to the user immersivefeedback and presentation device(s) are employed to increase theeffectiveness of loading the brain of the user. Feedback andpresentation are based on historical data derived from system 104 and/or106. Additionally, repeated playback of the data and information may beutilized to increase retention of the user receiving the data andinformation. In this manner historical and derived data and informationfrom system 100 is introduced to the user at a later time (T2, T3,T-nth) than recorded so that the information may be reiterated,retained, and operated upon by the brain and central nervous system.

Option E 294 b depicts the action of keeping a being alive by means of amechanical life support system which receives input from historical dataderived from a user 101 internal/external logging system 100. In thepresent context to emulate means to equal or surpass the cognitive orinformation processing abilities of the user 101.

FIG. 47 is a schematic diagram that illustrates Option D described inFIG. 46, which comprises stimulating a recipient 110 being's senses andbrain with historical data and information derived from the humaninternal/external logging system to restore or enhance memory andcognition. FIG. 47 illustrates data and information 287 derived from theportable LLEMA 100 system at time one T1 to the nth from user 101 isoperated upon to provide utility to a recipient 110 at time two T2 tothe nth by operating system 100. A dashed line 297 indicates an eventthat took place at an earlier time one Ti to the nth recorded by thesensor module 109. A dotted line 299 surrounding subject matterindicates an event is taking place at later time two T2 to the nth. Thesame system 100 operated at time one T1 to record, process, and log dataand information may be operated by a user at a later time two T2 torecall, process, transmit and input data and information for input to arecipient 110. Time one T1 and time two T2 refer to and may compriseeither an instance or period of time. For instance, Time 1 T1 and time 2T2 may be near real time (i.e. milliseconds) or very far apart in time(i.e. years). The oval shape in the drawing represents the brain 298 ofthe user 101. For example, in FIG. 47a user 101 at starting time one T1activates the portable system 104 sensor monitoring module 109 to recordbrain activity signatures and external sensor signatures of thesurrounding environment over a given timeframe. The signatures arestored as data and information 287 in the memory of computer 104 and/orremote computer server 106. The data and signatures from sensor module109 are read into memory are operated upon by the computer 104 and/orcomputer 106. Signature data and information is processed in thecorrelation module 111 of computer 104 and/or 106 to identifyrelationships and NCCs. A record of the NCC's identified along withsupporting data and information denoting the relationships between thedata and information is placed in and comprises a NCC database 288 whichis stored in the memory of computer 104 and/or remote computer server106. Supporting data and information preferably includes metadata thatrelates the derived NCC's information back to a filing system thatcomprises external signatures (i.e. video imagery signatures and audiosignatures, geo-spatial data, sub-vocalization data, etc.) and internalsignatures (i.e. brain activity and brain activity patterns). Theoperator of system 100, which may be the user 101, who operates aninterface device to set the parameters and metrics of the system 100that define thresholds of the data and information that define the NCCsof a specific user. System hardware and software designers willtypically construct the system 100, including the GUI, so that thesystem 104 and/or 106 is operable to achieve the results describedherein. The NCC database symbolizes the fundamental digitalrepresentation of how the user perceives the world.

Still referring to FIG. 47, computer 104 may incorporate various typesof filing systems and tags to organize the data. For instance, the datamay be filed or tagged by chronological, geographical, subject matter,or another method to facilitate memory storage and retrieval objectives.Once established the NCC database may be updated at any later time twoT2 to the nth by operating the system 104 and/or remote computer server106. The NCC database may be updated at follow-on data search engine andlogging instances. And check-points of NCC data may be stored andfollow-on data search engine and logging instances to create back-updatabases. Data and information recorded by system 104 may be offloadedover a wired or wireless network 105 to a remote computer system 106 forcomputer data and information storage and processing in order to reducethe demand on portable system 104. System 104 and 106 may be operated ina selective or continuous mode during the life of a user. Redundant,repetitive, or unwanted data and information may be filtered out throughcomputer processing at the time of recording or at a later time eitherautomatically, based on a rule set, or by a user or operator operatingsystem 100. User and operator querying and downloading the NCC database,a portion of the NCC database, information and data the NCC was derivedfrom, derivations from the NCC database, or an update to the NCCdatabase may be accomplished by using various database filing and querysystems known in the computer industry. For instance, related NCC dataand information may be stored on computer media familiar in the computerindustry in temporary and/or permanent memory of system 100. In thepresent example a dog 159 in the real world surrounding environment 160in which the user 101 is wearing system 100 senses, records, processes,and presents information. The system 100 acts on brain activitysignatures 163 that equate to the Conscious Percept (CP) 161representing the dog in the brain 167 of the user being 101. Informationrelated to the Conscious Percept of the dog is correlated withsignatures of a subject, activity, thought, or memory to derive anddefine the Neural Correlates of Consciousness (NCC) 166. The NCC, data,and information from which the NCC's are derived represent acomputerized relational database of information on how the user 101perceives the dog 159 and other percepts in a user's mind. At least someportion of the NCC and/or corresponding data and information, for say a“dog”, is passed on to a recipient 110 at time two T2.

The lower half of FIG. 47 illustrates an embodiment of the presentinvention in which at least some portion of the NCC database derived attime one Ti from user 101 is downloaded at a later time two T2 into thememory of a recipient 110. System 100 components and recipients 110 arebounded and defined by dashed line 299 in order assist in illustratingthe case example that takes place at time two T2. A recipient 110 in thepresent invention may be a living being 320 (i.e. a person 108), machine325 (i.e. a robot 286), or combination thereof (i.e. a bio-mechanicalsystem 319). The recipient 110 may be a user 101 who wore system 104 andlogged the data that is to be presented or a recipient who did not logthe data that is to be presented with the logged data and or informationderived from the logged data. Furthermore, recipients 110 may themselvesincorporate system 100 to create their own logged data and informationfor personal use or shared use with other recipients. And optionally, arecipient 110 user may be a clone of the user 101 whose appearance isidentical to the user who logged the data. Data and informationtransferred from system 104 and/or 105 to a recipient 110 may becorrelated or not correlated, depending on the design of the recipientwhom the information is to be input into. Data and/or information inputinto a recipient 110 is processed to be compatible and tailored forinput into that recipient.

For instance, in the lower portion of FIG. 47 capital letter V depictsan embodiment of the invention in which video imagery is output over awired or wireless telecommunication system and network 105 from outputmodule 296 of computer 104 and/or 106 at time two T2. The video signalis transmitted over communications local link 76 (i.e. wireless), 206(i.e. wired or fiber optical), or telecommunications system and networklink 105 for display to EMD's 137 a and 137 b worn by the recipient 110.Optionally and additionally, the audio portion of the video signal maybe transmitted to ear buds 138 a and 138 b previously described in thistext over either similar link 76, 105, or 206. The recipient 110 being320, here person 108, sees the imagery of a “dog” in the EMDs and hearsthe audio recordings of the “dog” barking in their ear buds. The imageryand audio presented from output system 296 at time two T2 is transducedby the eyes and ears that comprise a portion of the sensory system ofthe recipient 110. The audio visual content is transmitted from the eyesand ears of the recipient over the central nervous system to the brainwhere brain cells are stimulated or formed. In this manner forgotteninformation is presented to reinforce, restore, or create new braincells in the recipient 110 being 320.

The letter W illustrates another related embodiment similar to V.However, in embodiment V prior to presenting imagery and audio to therecipient 110 being 320, stem cells 313 are implanted and correspondingstem cell therapy is implemented. Stem cells for implantation receivableof the logged or derived information thereof according to the presentinvention may be created in vitro or in vivo. Implantation of the stemcells into a recipient being may be by any of a variety of methods toinclude brain surgery or injection well-known in the medical profession.In this manner, stem cell implantation can be used to replace, restore,and/or create new brain cells, and restore or create new memories usingdata and information derived from system 100. Once implanted,differentiation and migration of implanted stem cells is possible giventhe appropriate stem cell therapy. An oval shape illustrating braincells with a corresponding NCC signature overlaid over the recipient'shead in embodiment V and W illustrates that the video of the “dog”presented at time two T2 to the recipient is transduced to eitherreinforce, restore, or create the memory of the “dog” by eitherrestoring or creating new brain cells in the brain of the being 320.After stem cell implantation, cell therapy, and audio-visualpresentation of data derived from system 100 is presented, or, if onlyaudio-visual data derived from system 100 is presented, a test may beconducted to determine the presence or absence of the information of the“dog” in the memory of the user by operating system 100. Optionally,system 100 may be programmed to instantaneously call-up informationderived from the relational database, which includes the NCC database,to be presented to a user 101 or recipient 110 when it is detected bysystem 100 that the user or recipient cannot recall from memory a pieceof information derived from the past. Furthermore, given access, a useror recipient of the system 100 may receive information derived from theNCC database of another recipient or a plurality of recipients asdescribed earlier in FIG. 31.

Besides augmented cognition applications highlighted in the presentinvention, a final concluding objective is to enable beings to transfermore than just biological information forward by reproduction to auser's heirs and the rest of mankind. The data logged by individuals maybe operated upon for programming nanobots that may be introduced intothe brain to restore memory or introduce information into the neuralnetwork of the brain. Additionally, data logged from system 100 may beused in bio-engineering human systems that carry memories forwardthrough encoding those memories in our DNA and RNA. U.S. PatentPublication 2005/0053968, by Bharadwaj et al., dated 10 Mar. 2005, andtechniques disclosed in the UCD, Dublin, year 2012, publicationBioinformatics article entitled, “DNA Data Embedding Benchmark”, byDavid Haughton [30], that describes a system and method for embeddinginformation in the DNA string, while still preserving the biologicalmeaning of the string, is incorporated in full as a system and method ofa type which is integrated with the present invention to encode anddecode raw or correlated information derived from the present inventioninto human DNA. The logged information may include a test file, imagefile, or audio file in which large sequences are divided into multiplesegments and placed in DNA introduced to the user human or otherorganism. It is therefore an object to provide a system 100 that logs abeing's life experience such that a least some portions of the loggeddata may be codified and stored into DNA and RNA and passed to latergenerations as stored information in a living organism, a cadaver, ortransferred to another living being though reproduction. DNA withencoded information derived from the present invention is implanted intoa fertile egg or sperm of a human, embryo, or fetus, to transfer theinformation genetically using medical procedures familiar to thoseskilled in the art. For instance, an image of a being's ancestors couldbe carried forward in the DNA of the being so that the being couldaccess the image in order to see the being they evolved from. In thismanner, a human may transcend or pass on his experience in the form ofhis memories and the lessons he or she learns throughout life. Much ofthe information that comprises the individual essence of a person'sconsciousness, including thinking processes, experiences, and memory, islost because of human mortality. The present invention may be used tohelp overcome that limitation by recording, storing, and reloadinglogged data into a post predecessor specimen. It is therefore conceivedin the present invention that nanobots may be programmed with datalogged into and derived from the present video logging and enhancementsystem. It is also conceived in the present invention that data loggedinto and derived from the present video logging and enhancement systemmay be coded into genetic DNA or RNA which may be passed viareproduction into offspring or implanted into other individuals. Aperson's experiences are the memories and connections that personconstructs as that person journeys through life. This invention allows aperson, and hence mankind, to carry forth that journey with decreasedloss of information and consciousness.

The letter X illustrates yet another embodiment of the invention.Embodiment X represent a system and method wherein at least some portionof data and/or information 287 and/or 288 derived from system 100 isstored on DNA 322. Specimens of DNA 321 may be taken from the body ofthe user 101 or recipient who is a biological 320 or bio-mechanical 319being. For instance DNA specimens may be from the skin or hair of therecipient 110. Data logged from system 100 may be used inbio-engineering human systems that carry memories forward throughencoding those memories in our DNA and RNA. U.S. Patent Publication2005/0053968, by Bharadwaj et al., dated 10 Mar. 2005, and techniquesdisclosed in the UCD, Dublin, year 2012, publication Bioinformaticsarticle entitled, “DNA Data Embedding Benchmark”, by David Haughton[25],that describes a system and method for embedding information in the DNAstring while still preserving the biological meaning of the string; isincorporated in full as a system and method of a type which isintegrated with the present invention to encode and decode raw orcorrelated information derived from the present invention into humanDNA. The logged information may include a test file, image file, oraudio file that in which large sequences are divided into multiplesegments and placed in DNA introduced to the user human or otherorganism. It is therefore an object to provide a system 100 that logs abeing's life experience such that a least some portions of the loggeddata may be codified and stored into DNA and RNA and passed to a latergenerations, as stored information in a living organism, a cadaver, ortransfer to another living being though sexual or artificialreproduction. DNA with encoded information derived from the presentinvention is implanted into a fertile egg or sperm of a human, embryo,or fetes, to transfer the information genetically using medicalprocedures familiar to those skilled in the art.

It will also be understood to those skilled in the art that embeddinginformation in DNA can be carried out in vitro or in vivo. For instancein vivo at least some portion of data and/or information 287 and/or 288derived from system 100 stored on DNA may be implanted surgically or byinjection using known medical procedures. The method for storinginformation in DNA includes software which takes information derivedfrom system 100 that is in ASCII machine 323 language represented as along string of ones and zeros in computer code. Then a computer programconverts this code into letters A, C, G, and T, which correspond to thefour chemical bases that make up DNA. The program breaks up the longstring of letters and indexes them. A machine uses that data to makeDNA. The DNA goes into a sequencing machine, which reads back the DNAfragments as the letters A, C, G and T. A computer program reassemblesthe DNA fragments in the correct order, and converts them back into onesand zeros. A computer interprets the ones and zeros as the originalinformation and plays it back on the computer. The ASCII character setused in this example may be encrypted, stored, and then decrypted astext, audio, imagery (including video), and etc. Logged data andinformation from system 100 that is embedded on DNA 313 may include theraw and/or derived information by operating system 100. Data embeddedDNA is a good storage mechanism under suitable conditions because whendecrypted data embedded DNA has near perfect fidelity with little dataloss (i.e. >99.9%), has a long storage life (i.e. hundreds of thousandsof years), large amounts of data can be stored while taking up verylittle space (i.e. terabits of information within the size of severaldust particles), and may be stored within the DNA of living cells or onnon-living matter. Alternatively, it will be understood by those skilledin the art that DNA embedding may be carried out in vitro wherein thesampling and processing of tissue coding and decoding process takesplace on the recipient on implanted computational modules like thatshown in FIG. 16 and sampling of tissue for DNA embedding accomplishedby servo devices 181 shown in FIG. 15.

Each DNA segment represents a processor to execute a particularbiological process for growth and maintaining the life the hostbiological being. Embedding knowledge derived by humans outside thenatural information inherent within DNA provides a method and tool forpassing additional information between biological 320, bio-mechanical319, or mechanical 286 recipient 110. Embedding information within DNAderived by humans outside the natural information inherent within DNAalso provides a method and tool for passing additional information fromgeneration to generation via natural reproduction or electronicallydepending on the specific design and make-up of the recipient.

The letter Y illustrates yet another embodiment of the invention.Embodiment Y represent a system and method wherein at least some portionof data and/or information 287 and/or 288 derived from system 100 isinput into a recipient 110 by implanting or injecting at least onenanobot 324. The nanobot may serve as a delivery system for derivedinformation to cells within the brain. Alternatively, the nanobot maycarry stem cells to an area of the brain that are stimulated asdescribed in embodiment W. Or alternatively, the nanobot may deliver DNAto a location as described in embodiment X. Or still alternatively, thenanobot can deliver electronic devices, like the diagnostic devicedescribed in FIG. 48, within the brain.

The letter Z illustrates yet another embodiment of the invention.Embodiment Y represent a system and method wherein at least some portionof data and/or information 287 and/or 288 derived from system 100 isinput into a recipient 110 which is a machine 286 by directly inputtingcomputer code 258, 259 into a recipient. Input of the information may beaccomplished via a ground (i.e. wire or fiber-optic) or over-the-air(i.e. radio frequency or infrared) connection.

Still referring to FIG. 47, each original user 101 has a unique set ofthe experiences and related brain activity that can be added to by eachsubsequent recipient 110. The derived database 287, including the NCCdatabase 288 provides a record unique to each user 101. Transfer of thederived relational database 287 to include historical data on CPs andNCCs unique to an individual user may be transferred or communicated bya previous user 101 prior to death or upon death to a recipient 110using the methods and systems described in FIG. 47. For a user to passat least some portion of his or her database 287 to a recipient, theuser would need to arrange for such information to be transferred byanother user or in an automated manner to a recipient upon death. Insuch an instance, a signal from the system 104 could be sent over thetelecommunication system 105 to a recipient who has the ability to beactivated upon death or incapacity of the original user. Because theuser may not know when life will be terminated updates to the database287 could be constantly stored in memory on a remote server 106. In thismanner data and information derived from the invention could be storedin memory by a user at an earlier time/check-point (i.e. at T1) and beretrieved by a designated recipient at a later time/checkpoint (i.e.T2). In this manner, a user's awareness would continue, albeit in adifferent being or machine than prior to the parent user's terminationor incapacity. The database 287 passed is normalized and translated to atype compatible with the recipient. As long as the entire system 100remained activated, remote system 105 could pass its logged database 287of a user 101 to a recipient 110 based on a given activity or after aperiod of inactivity of the user 110 so that the sentience of the user101 is passed on to the recipient 110. It is worth noting that therecipient would possess the ability to recognize the corpse of adeceased user as his or her previous body if the CP and NCC's weresuccessfully communicated to the recipient. Correspondingly, a recipientwould possess the ability to recognize a living user as his or herprevious body if the CP and NCC's were successfully communicated to therecipient. But the living user would not necessarily recognize theoutward appearance of the user. Each user's experience and related brainactivity is based on what is logged, processed, and input. The degree towhich a being 101 or 110, machine 286, or biomechanical being 319incorporates the information derived from the present invention will bedetermined by the quality and quantity designed into each givenembodiment of the present invention 100. Filtering out superfluousinformation and timely relevant delivery of derived information from thepresent invention is a key part in defining the fidelity of the derivedrelational database 287 and relationship to the original user. Thedegree to which the recipient is able to incorporate the informationderived from the present invention will determine the qualitative andquantitative contentment and longevity of a user's life as a recipient110 (i.e. 320, 319, or 286) in a given environment.

FIG. 48 is a schematic diagram illustrating at least one diagnosticmicrochip 300 and at least one stem cell 313 implanted into a recipient110. The microchip has the operational capability to sense neuralactivity, including activity sensed from an implanted stem cell, in aspecific area of the brain 167. The microchip 300 incorporates an AMRsensor 301 of a type referenced earlier by Kitching. A solid line witharrows indicates the magnetic field strength 315 the magnetometerregisters of the brain activity. The microchip includes a computerprogrammable central processing unit (CPU) 302 that controls componentson the microchip. The microchip includes a computer memory 304 storagecapability common to computer devices. Microchip components include atransceiver 305 that receives and transmits radio-frequency (RF) signals307 to a remote transceiver 172. The remote transceiver functions as aninput and control device 296 for the microchip and receiver of datasensed by the atomic based sensor. The remote transceiver and computerinput device may be incorporated into a computer system, such as asmartphone. In the present example, a kinetic power generator 306 isintegrated into the microchip so that the microchip can function withoutphysical connections such as wires or fiber optics conduits running frominside to outside the skull 13. It will be known to those skilled in theart that various power generation systems may be incorporated, toinclude wired and wireless power transmission systems. The powergeneration system may include a storage battery 21. Transceiver andpower generations systems may be commanded to be turned on and off inorder to alleviate interference to the AMB sensor. Alternatively, andadditionally, an electrical stimulation electrode 303 is incorporatedinto the microchip. The purpose of the electrode is to stimulate anadjacent area of the brain in proximity to the microchip. The electricalsignal emitted from the electrode is indicated by a solid arrow 314. Theelectrical stimulation is done by the electrode emitting an electricalsignal into the brain. The electrical signal may be of the area wherestem cells have or have not been transplanted. The purpose ofstimulating a specific area is to evoke a response of the recipient userof the implanted microchip. The microchip is implanted using surgicalprocedures well known in the medical profession. These includesterilization of the microchip and/or protective covering that maysurround the microchip. The components of the microchip are connectedtogether by circuitry running through or on conventional substratamaterial 317 of which microchips are constructed.

For example, in FIG. 48, assume a user has Alzheimer's and experiencesmemory loose in at least one section of the brain 167. Clinical analysisis conducted to determine what information the recipient patient hasforgotten from his or her memory. For instance, the patient may be askedto recall the information at time two T2 that happened earlier at timeone T1. Evidence that the information has been forgotten may be that therecipient patient responds that he or she does not remember theinformation. This can be tested by asking the recipient if he or she isable to recall the information and verbalize the forgotten information.Additionally and alternatively, the location in the brain where memoryloss occurred may be determined by looking at brain activity recordslogged by the system 100. Evidence that the information is no longerresident in the memory of the recipient patient is that brain activityof the recipient patient does not react at all or even similarly to howthe brain activity reacted historically at time 1 T1 when similarsubject matter was presented to the user. In such an instance, theminimum criteria defining a particular NCC focus representing a stimulusthat defines the particular subject matter at time one T1 will not reactsimilarly to a like stimulus at time two T2. This can be deduced bycomparing the T2 response with the T1 response recorded in the system100 relational database 287, which includes the NCC database 288. Fromthis clinical analysis regions, nucleus, synapse and so forth in therecipient's brain may be identified and targeted for stem cell therapyimplantation.

Still referring to the present example, illustrated by FIG. 48, theright eye 225 a and the left eye 225 b of the recipient of the stemcells includes corresponding EMD's 137 a and 137 b. Existing brain cells316 are left in place, but inactive brain cells determined to bedamaged, and inactive brain cells in the targeted area 310 of the brainaffected by Alzheimer's disease, are surgically removed and/or replacedby the introduction of implanted stem cells 313 into the brain. Theimplanted stem cells 313 form newly created brain cells 311 as loggedinformation from system 100 that has been identified as lost memory ofthe recipient 110 patient is recalled, replayed, input, and transmittedto the wireless EMD's and earphones (not shown) the recipient 110patient is wearing. A dashed line with arrows at each end indicates theradio frequency (RF) transmission signal 307 between the microchip 300transceiver 305 and the input device 296 transceiver 172. Dashed linesbetween the eyes and implanted stem cells indicate the transmission ofthe input information 309 to stem cells, which become the newly createdbrain cells with the restored information. The drawing of the “cat”indicates the input subject matter 308 transmitted for restoration ofthe memory lost by the recipient 110 patient. In the present example,the Conscious Percept of the user 101 is derived from a real-worldsubject 161 comprising a “cat”. At least some portion of the NCC and/orcorresponding data and information, for say a “cat”, is passed on to arecipient 110 at time two T2 when information is transmitted to theimplanted stem cells that form the newly created brain cells 311 andrestored memory 312 in the brain cells that represent the “cat”.

Alternatively, the microchip 300 incorporating an atomic-based magnetic(AMB) sensor 301 system of a type referenced earlier by Kitching forsensing brain activity may be mounted on the exterior of the user. Aplurality of sensors may be located in the covering worn by the user andreadout may be in wired or wireless communication to a computer system.The host computer system may be designed into the covering or locatedremotely on or away from the user's body. The A skull cap, wig, or caparrangements previously described may be incorporated to hold one ormore AMB sensors in order to provide a non-invasive method of monitoringbrain activity of the user or recipient patient.

FIG. 49 is a schematic diagram illustrating Option D 294 b described inFIG. 46. FIG. 49 is a discloses a life support system that uses therelational database 287 derived from internal and external data loggingsystem 100 according to the present invention. Applying the historicaldatabase to operation of the life support system can assist physiciansin achieving homeostasis to a recipient. The objective being to bringthe patient back to a stable condition like that at shown by historicalphysiological data collected by system 100 about a user 101 when he orshe was healthy and stable. Life support apparatus such as feedingtubes, total parenteral nutrition, mechanical ventilation, heart/lungbypass, urinary catheterization, dialysis, cardiopulmonaryresuscitation, defibrillation, artificial pacemaker, artificialrespiration, neural stimulation and mind uploading devices are operatedto keep a patient living. Live and historical databases derived fromcomputer 104 and/or 106 and associated sensor systems result in aninternal sensor derived database 287 from MM, biometric, physiologicalsensors and an external sensors derived database from Visual, Audio,Smell, Touch, Taste sensors are sampled to maintain or restore a beingor machines physiological, homeostasis, memory, and cognitive functions.As represented by the arrows between the Live and Historical Databaseand the Life Support Apparatus the logged data from system 100 is inputinto the life support systems to set the parameters the life supportsystems. The life support apparatus is then hooked to a user's body asindicated by the arrows to the separated body parts. The separated bodyparts may be maintained individually or when reattached to one anotherby medical professionals. As indicated by the arrows the live andhistorical databases, life support apparatus, and human body partsinformation may flow both directions to regulate and maintain the humanbody part(s). Besides the life support system operating on data andinformation 287 derived from the system 100 that was or is born by theuser 101, data and information from other users 102 and recipients 110who have had successful treatments of a pathology may be implementedinto the life support system of user 101 to assist in curing him or her.In this manner, parameters tailored to a user may be introduced to userwhen he or she is on a life support system. Historical data derived fromthe data logging and memory enhancement system is used as a life supportand restoration system.

Additionally, it is conceived as part of the present invention thatclinical and non-clinical records and specimens from a user's body,while in vivo or envois, may be used to further replicate a user 101.For instance, skin and hair may be used to replicate the DNA sequence ofthe user in order to reproduce a detailed replication of the user. Orfor instance, once a person is deceased it is envisioned as part of thepresent invention that the user's body may be frozen and sliced, imaged,and modeled in 3-D to form a very detailed computer simulated model ofthe user's entire body. Further still, besides the data recorded by theinternal logging and memory enhancement system the user wears,additional internal data, such as three-dimensional full body MMcomputer models and data may be added to the logged database 287. It isconceived as part of the present invention that this data may beincorporated into various simulation, augmentation, diagnostic,substitution, restoration, and emulation devices, processes, andprocedures during and after the life of the user.

The invention is preferably implemented by hardware, software, andbiological specimens, or a combination of hardware and software andbiological specimens. The software can be embodied as computer readablecode on a computer readable medium. The computer readable medium is anydata storage device that can store data which can thereafter be read bya computer system. Examples of the computer readable medium includeread-only memory, random-access memory, CD-ROMs, DVDs, magnetic tape,optical data storage devices, and carrier waves. The computer readablemedium can also be distributed over network-coupled computer systems sothat the computer readable code is stored and executed in a distributedfashion. Thus, it is seen that systems and methods are provided forallowing users to couple a portable electronic device in thehead-mounted device. It is also seen that systems and methods areprovided for allowing users to see the outside world while wearing ahead-mounted device. Persons skilled in the art will appreciate that theinvention can be practiced by other than the described embodiments,which are presented for purposes of illustration and not of limitation,and the present invention is limited only by the claims which follow.

Although the invention has been described with reference to theembodiments illustrated in the attached drawing figures, it is notedthat equivalents may be employed, and substitutions made herein, withoutdeparting from the scope of the invention as recited in the claims.

Having thus described various embodiments of the invention, what isclaimed as new and desired to be protected by Letters Patent includesthe following:
 1. A user conveyable system comprising: a brain activitysensing subsystem configured to collect data corresponding to brainactivity of a user; a measurement computer subsystem configured togenerate data representing quantifications of perceptions of said user;a user sensing subsystem configured to collect data corresponding touser events; a surrounding environment sensing subsystem configured tocollect data corresponding to a user's surrounding environment; arecording subsystem configured to record said data from said brainactivity sensing subsystem, measurement computer subsystem, user sensingsubsystem, and surrounding environment sensing subsystem; a correlationsubsystem configured to create correlation subsystem data, saidcorrelation subsystem data comprising relationships between said datacorresponding to said brain activity of said user, said datacorresponding to said user events and said data corresponding to saiduser's surrounding environment; a user mobile electronic device incommunication with said brain activity sensing subsystem, measurementcomputer subsystem, user sensing subsystem, surrounding environmentsensing subsystem, and recording subsystem, said user mobile electronicdevice including an interactive graphic user interface and beingconfigured to: operate as a host computer processing subsystem forcommand, control, and processing of signals to and from said brainactivity sensing subsystem, user sensing subsystem, surroundingenvironment sensing subsystem, and correlation subsystem; command saidbrain activity sensing subsystem to transmit brain activity and patterndata to said correlation subsystem; and command said user sensingsubsystem and surrounding environment sensing subsystem to transmitprocessed sensor data to said correlation subsystem, said correlationsubsystem being configured to receive and perform correlation processingoperations to determine an extent of neural relationships between datareceived from said user mobile electronic device and said brain activitysensing subsystem, user sensing subsystem, and surrounding environmentsensing subsystem to derive neural correlates of consciousness ofconscious precepts of said user; a non-transitory computer readablemedium configured to store data from said brain activity sensingsubsystem, measurement computer subsystem, user sensing subsystem,surrounding environment sensing subsystem, recording subsystem, andcorrelation subsystem for performing queries on real-time and nearreal-time data received from said brain activity sensing subsystem,measurement computer subsystem, user sensing subsystem, surroundingenvironment sensing subsystem, recording subsystem, and correlationsubsystem for determining whether to keep or disregard said data fromsaid brain activity sensing subsystem, measurement computer subsystem,user sensing subsystem, surrounding environment sensing subsystem,recording subsystem, and correlation subsystem based on pre-establishedrule-sets and user interactive command and control from said user mobileelectronic device; and a computer processing device configured toprocess and communicate at least a portion of said data from said brainactivity sensing subsystem, measurement computer subsystem, user sensingsubsystem, surrounding environment sensing subsystem, recordingsubsystem, and correlation subsystem into at least one of a said userconveyable system, peripheral or remote subsystem, or a recipientbiological, bio-mechatronic, or mechatronic system.
 2. The userconveyable system according to claim 1, wherein said user conveyablesystem further comprises: said recipient biological or bio-mechatronicsystem that includes at least one artificial synaptic chip that receivesdata derived by said user conveyable system, peripheral or remotesubsystem, or said recipient biological, bio-mechatronic, or mechatronicsystem.
 3. The user conveyable system according to claim 1, wherein saiduser conveyable system further comprises: an integrated near-eye displaywith camera assembly positioned forward of said user's face and held bya support structure mounted on said user's head which cooperativelydisplays images to at least one eye of said user by at least onesee-through curved mirrored combiner, waveguide, or OLED based opticalhead mounted display system; and an integrated panoramic camera systemcollocated with the viewing surface of said near-eye display; saidcamera system including a plurality of off-axis optical elements some ofwhich are arranged to look inward at the face and eyes of said user andadditional optical elements which are arranged to look outward to recordthe remainder of a panoramic scene to the sides and forward the face ofsaid user; said panoramic camera system integrally collocated with thenear-eye display configured to at least not block said user's view of animage emitted from the display into said user's eyes, and to not blockreflected image projection off a mirrored or semi-mirrored or otherwaveguide surface that originates from a display into said user's eyes,and to not block said user's view of the real world beyond the displayedor projected image.
 4. The user conveyable system according to claim 1,wherein said user conveyable system is configured to input at least someportion of user data logged or derived by said user conveyable systeminto a computer generated human simulation that utilizes at least oneartificial intelligence or artificial intelligence-like computerprocessing capability.
 5. The user conveyable system according to claim1, wherein: said user conveyable system comprises a basic component forproviding a user's, group of user's, recipient's, or group ofrecipient's data input into a computerized training management tool, thecomputerized training management tool including a live, virtual, gaming,or constructive computer training simulation system, or a real-worldun-simulated environment monitored to record and receive at least someportion of user data logged or derived from said user conveyable system;said computerized training management tool operating to measure theperformance of a user, group of users, recipient, or group of recipientsin the simulation system, or a user, group of users, recipient, or groupof recipients wearing said user conveyable system in a real-worldun-simulated environment; said computerized training management toolincluding: a comparator computerized processing module to calculate andcompare baseline performance results comprising iterative operations ofusers or recipients operating in a preceding simulated trainingsimulation scenario, or an un-simulated real-world situation, with theresults of users or recipients operating a subsequent simulated trainingsimulation system scenario, or un-simulated real-world environment inwhich changes are made in at least one time, space, subject matter, oractivity that affect the user or recipient, or the user's or recipient'senvironment, to determine the variables that improved, left unchanged,or did not improve the baseline performance of said user, group ofusers, recipient, or group of recipients; then affecting the surroundingenvironment or functional variables of at least one user or recipient inat least one live, virtual, gaming, or constructive computer trainingsimulations, or a real-world un-simulated environment, consistent withthe performance of said user, group of users, recipient, or group ofrecipients, with the intention of achieving a different or desiredoutcome.
 6. The user conveyable system according to claim 1, whereinsaid user conveyable system comprises: a housing enveloping amicroprocessor that is implanted into the brain of a user; saidimplanted microprocessor including: an integrated brain stimulationelement, said brain stimulation element including at least one mechanismto stimulate the brain in a desired area to achieve a result; a brainactivity sensing and measurement system, said sensing and measurementsystem sensing brain activity; a wireless communication system element,said wireless communication system element including a radio frequencytransceiver for communication with a remote device for exchanging dataand controlling one or more of said implanted microprocessors; and/or anonboard regenerative electrical power source; said implantedmicroprocessor or microprocessors being controlled by an operator todiagnose, treat, or function within a desired area of the brain of saidbiological or bio-mechatronic system.
 7. The user conveyable systemaccording to claim 1, wherein said user conveyable system comprises: ananobot system including at least one nanobot including a brain activitysensing subsystem, a wireless communication subsystem, and a brainstimulation subsystem; said nanobot being injectable or implantable;said nanobot being user positional in the brain; said nanobot includinga sensing and measurement subsystem sensing brain activity; said nanobotincluding an onboard regenerative electrical power source; said nanobotincluding at least one of a substance transport electrical,electromagnetic discharge, light, or ultrasonic device to stimulate thebrain in a desired area to achieve a result; and said communicationsystem including a radio frequency transceiver for communication with aremote device to receive and send information for controlling one ormore implanted nanobots; said nanobot being controlled by an operator todiagnose, treat, and/or function within a desired area of the brain ofat least one of said biological or bio-mechatronic systems.
 8. The userconveyable system according to claim 1, wherein said user conveyablesystem comprises: a user mobile electronic device that is a smartphonethat fits into a head mounted support structure mounted on the user'shead that operates as a near-eye panoramic display system that the uservisually observes and interacts with as part of the said interactivegraphic user interface.
 9. The user conveyable system according to claim1, wherein said user conveyable system includes: a video camera anddisplay subsystem comprising at least one of an audio sensor, imagingsensor, and electroluminescent display assembly affixed by at least oneof a support, magnetic, prosthetic, adhesive, or a piercing assemblysupported by a mount on the body of a user, or on something worn on saidbody, or partially implanted in said body, and separable about the user;each said sensor and display assembly in a communicating relationship tosaid user conveyable system by a wireless transceiver for fast fusioncomputer processing to facilitate computerized digital video analysis,and processing and presentation by said user conveyable system; eachsaid video camera and display subsystem including a portable electricalpower source; said conveyable video camera and display subsystemfacilitating the capture and presentation of information drawn from theuser conveyable system.
 10. The user conveyable system according toclaim 1, wherein said user conveyable system comprises: an integratedvideo camera and electroluminescent display system that operates as auser and environmental sensing system and graphic user interface for auser and a vehicle; said video camera and electroluminescent displaycomprising the interior covering of a vehicle that substantiallysurrounds or is worn by a user, and the video camera andelectroluminescent display system that comprises the outer covering ofat least some portion of said vehicle; said user, vehicle, andintegrated video camera and electroluminescent display system in acommunicating relationship with a graphic user interface and a commandand control system of said user conveyable system to facilitate at leastone land, sea, air and space operation and navigation of said vehiclethat is autonomously or semi-autonomously piloted; said covering beingaffixed to said vehicle; said user conveyable system being powered by avehicle portable electrical power system.
 11. The user conveyable systemaccording to claim 1, wherein said user conveyable subsystems anddevices take action based on system pre-established rule-sets and userinteractive command and control preferences incorporated within at leastone of a said user conveyable system, peripheral subsystem, remotesubsystem, or a recipient biological, bio-mechatronic, or mechatronicsystem to conduct user and recipient queries, perform context basedservices, and perform client services for mobile devices over atelecommunication network.
 12. The user conveyable system according toclaim 1, wherein said brain activity sensing subsystem comprises: atleast one artificial synaptic chip implanted in the brain of said userthat provides a brain to computer interface and a computer to braininterface, or a combination thereof; wherein said brain to computerinterface is operated by said user conveyable system to sense, record,and store signatures communicated from said artificial synaptic chip asdata about a particular cell structure representing a specific neuralarea related to a specific neural correlate of consciousness orcomponents thereof expressive of a conscious percept stored as part ofsaid data received, processed and stored by said user conveyable system;wherein said computer to brain interface operates in real-time or nearreal-time on logged data or data derived from said synaptic chip tocontrol a brain stimulation device to stimulate said user or a remoteuser of the system, or a recipient biological, mechanical, orbio-mechanical system; said user, remote user, recipient biological,mechatronic, or bio-mechatronic systems stimulated by a procedure andsaid brain stimulation device that administers an electrical, light,sound wave or chemical stimulation to a neurite or cell body connectedto said artificial synaptic chip that represents a particular cellstructure representing a specific neural area related to a correspondingspecific neural correlate of consciousness or a component thereofderived from a conscious percept that is part of said stored data;wherein said user conveyable system operates to conduct both input andoutput between said artificial synaptic chip and said stored datareceived, processed and stored by said user conveyable system or arecipient biological, mechatronic, or bio-mechatronic system.
 13. Theuser conveyable system according to claim 1, wherein said userconveyable system comprises: a vehicle for land, sea, air or spacenavigation that includes a computer navigation system with an artificialneural network that operates on at least some portion of incoming data,historical data, data queried, or data derived from said: a) userconveyable system, b) recipient biological system, c) user peripheralsubsystem, d) remote subsystem, e) recipient mechatronic system, or f)recipient bio-mechatronic system, to semi-autonomously or autonomouslyoperate said vehicle.
 14. The user conveyable system according to claim1, wherein said user conveyable system comprises: a target designationand tracking system that operates on said data received or derived froma user conveyable system, a user peripheral or remote subsystem, or arecipient biological, mechatronic, or bio-mechatronic system.
 15. Theuser conveyable system according to claim 1, wherein said userconveyable system communicates at least a portion of incoming data,historical data, or derived data to a quantum computer system thatcomprises at least one part of said user conveyable system, a remotepart of said user conveyable system, or a part of the recipientbiological, bio-mechatronic, or mechatronic part of said user conveyablesystem.
 16. The user conveyable system according to claim 1, whereinsaid user conveyable system comprises: a plurality of user conveyablesystem neural correlates of consciousness databases, and/or a pluralityof recipient conveyable system neural correlates of consciousnessdatabases; each user and recipient conveyable system neural correlatesof consciousness database comprising the data sensed and/or the dataderived from at least one of: a) the relationships within the brain, orb) the relationships between the brain and surrounding environment,either or both of which defines a user or recipient system's neuralcorrelates of consciousness data; said user data and said recipient dataprocessed in a computer environment; wherein selected said user data andselected said recipient data is processed in said computer environmentto normalize said user data and said recipient data to define a commontaxonomy between said user data and said recipient user; and thetaxonomy operated upon as required for translation into computerlanguages to facilitate communication between a plurality of users; saidplurality of users including either users of a said user conveyablesystem, peripheral or remote subsystem, or a recipient biological,bio-mechatronic or mechatronic system.
 17. The user conveyable systemaccording to claim 1, wherein said user conveyable system includes: abrain activity sensing subsystem that is part of the user conveyablesystem, including an infrared electroluminescent display that facesinward about at least some portion of the head of a user; said brainactivity sensing subsystem including a sensing subsystem, surroundingenvironment sensing subsystem, and correlation subsystem; said infraredelectroluminescent display of a frequency band that focuses thedisplayed light that faces inward on at least one voxel of the user'sbrain tissue and a near-infrared sensor that senses a return orreflection of the light in a frequency band to capture an image that iscorrelated to a Neural Correlate of Consciousness (NCC) associated witha certain conscious percept (CP) of said user to assist in performingaugmented cognition for said user negotiating a space, or operating avehicle; the system and subsystem components being in communicatingrelationship with one another and other subsystems that are part of theuser conveyable system; and said system and subsystems powered by aportable electrical power generation system.
 18. The user conveyablesystem according to claim 1, wherein said user conveyable systemcomprises: at least one of an imaging, auditory, olfactory, haptic,tasting, and cognitive sensing system; said imaging, auditory,olfactory, haptic, tasting, and/or cognitive sensing system includingrecording and feedback components for affecting said entity's self or arecipient system.
 19. The user conveyable system according to claim 1,wherein said user conveyable system comprises: stem cells and progenitorcells of said mobile user and recipient biological and bio-mechatronicsystems targeted so that at least a portion of said data derived fromsaid brain activity sensing subsystem, measurement computer subsystem,user sensing subsystem, surrounding environment sensing subsystem,recording subsystem, correlation subsystem, and data generated for inputor derived from a recipient system is introduced into the brain; whereintargeted stem cells or progenitor cells are at least one implantedsurgically, implanted by injection, are neural stem cells that areself-renewing, or are progenitor cells; said data generated by saidmobile user and recipient system being introduced by stimulating theuser or recipients senses externally via the central nervous system orsensor system of the user or recipients with media logged by said mobileuser or recipient user data and surrounding environment data correlationsystem; or said data generated by said mobile user and recipient systembeing introduced to stem cells implanted or renewed by loadinginformation into said user or recipient user via a nanobot implanted inthe brain, MEM, or a synaptic chip implanted in the brain targeted atimparting derived data to at least one stem cell or progenitor cell insaid user of said system or a said recipient of said system.
 20. Theuser conveyable system according to claim 1 wherein said brain activitysensing subsystem comprises: an infrared signal generation system whichis focused onto at least one voxel in the brain at a given wavelength; arecording system that repeatedly records consecutive exit signals ofinfrared signal generated; said brain activity system includingcomparator logic operatively comparing the exit signals to one anotherto derive the location of at least one said voxel in the brain; and acorrelation computer subsystem operated to iteratively analyze saidbrain activity signals to identify the optimum imaging signalrepresenting a voxel or voxels corresponding to new or previouslyidentified neural correlates of consciousness related to data andmetadata that is stored in a non-transitory computer readable medium inthe memory of said user conveyable system; said infrared raw orprocessed signals operated upon by a computer system or computersubsystem as part of at least one of the said user system, a recipientbiological, bio-mechatronic, or mechatronic system, or directing asecond brain activity sensing subsystem to the same location in thebrain of a biological or bio-mechanical system for diagnostic ortherapeutic purposes.
 21. The user conveyable system according to claim1, wherein: said user conveyable system comprises computer and sensingsubsystems from which neural correlates of consciousness are generated,formulated into perceptions of the user, stored, weighed, and acted uponby a computer processing device utilizing thresholds and rule-setsdefined by at least one a user, the user system employing an artificialneural network, an agent of said user, or a system administrator that atleast one filters, inhibits, or blocks certain inappropriate output frombeing actioned, and certain data and information derived by said systemfrom being released to at least one of another user, peripheral orremote subsystem, or recipient biological, bio-mechatronic, ormechatronic system in order to guard against the release of random andpersonal thoughts, and to protect the rights, privacy, and security ofthe user or other beings and things in the surrounding environment. 22.The user conveyable system according to claim 1, wherein: said userconveyable system, peripheral or remote subsystem, or recipientbiological, bio-mechatronic, or mechatronic system includes historicalneural activity data and/or surrounding environment data that isrecorded and correlated and then compared to subsequent neural activitydata and/or said surrounding environment data that is recorded andcorrelated in order to predict the likelihood of future actions beingtaken by at least one of said systems or subsystems; said predictiveresults stored in memory and operated upon to affect future neuraland/or surrounding environment activity of at least one of said systemsor subsystems with the intention of achieving a similar, different, ordesired outcome.
 23. A self-reliant entity comprising: a) a conveyablecomputer system with a cognitive memory system and computer subsystemsoperated in real time to dynamically correlate neural network activitydata with surrounding environment data akin to a human: (1) a firstcomputer subsystem including and compatible with said conveyablecomputer system that derives, includes, and is compatible with a neuralcorrelates of consciousness database stored as non-volatile cognitivememory in said computer subsystem that defines a taxonomy for saidself-reliant entity's perception of self and the environment; (2) saidfirst computer subsystem including and compatible with said neuralcorrelates of consciousness database operated upon by at least one abiological neural network and artificial neural network that includesbackpropagation processes that iteratively and adaptively derive logicbased outcomes that improve results or achieve desired outcomes that aredecided by said self-reliant entity and determine said self-reliantentity's activity; (3) said first computer subsystem including andcompatible with said self-reliant entity's sensor arrangement thatoperates to record and store in non-volatile memory entity self-sensingand surrounding environment image signatures, operates on non-volatilememory of image signatures to formulate a plan based on saidself-reliant entity's internal operations and the external surroundingenvironment, and acts with the intent to reach a goal based on the planderived and shaped by said self-reliant entity's overall design; (4)said first computer subsystem operating on at least one neural networkto derive neural correlates of consciousness from conscious percepts andsubsequently to operate on those neural correlates of consciousness tomake decisions as a self-reliant recipient system; (5) a second computersubsystem including an actuator and manipulator subsystem that operateswith the mobility and dexterity akin to a human; (6) a third computersubsystem including a rechargeable energy generation subsystem thatoperates akin to a human; b) said conveyable computer system andsubsystems, and energy generation, actuator and manipulator subsystems,being in communicating relationship, and once initiated operating as acohesive system akin to a human.
 24. The self-reliant entity accordingto claim 23, wherein said user self-reliant entity comprises: aconveyable computer system with a computer subsystem that incorporates acognitive memory and artificial auto-associative neural network basedsearch engine for computer and network located images, audio andphotographs.
 25. The self-reliant entity according to claim 23, whereinsaid user self-reliant entity comprises a host and recipient system thathas the intent to plan and act to design, construct and maintain atleast one of itself, and like entities, perpetually based upon at leastone data from, and data created by, said self-reliant entity.
 26. Theself-reliant entity according to claim 23, wherein said userself-reliant entity comprises: a rechargeable electric energy generationsubsystem including at least one of: a) a rechargeable battery, b) aninductive battery charging system, c) a photovoltaic assembly with aphotovoltaic light sensitive exterior covering, or d) a kinetic energygeneration assembly.
 27. The self-reliant entity according to claim 23,wherein said self-reliant entity includes computer logic, which operatesto allocate computer processing tasks, such that short-term memory andfrequent cognitive tasks are processed onboard said self-reliant entity,and at least one of the following tasks, including: a) long term memorystorage, retrieval, and processing, b) agent work, c) intensive, deeplearning, d) searches, e) queries, f) security screening, and g) massstorage are off-loaded from the self-reliant entity to at least oneremote computing device on a telecommunications network that iscompatible with and in a communicating relationship with saidself-reliant entity.
 28. The self-reliant entity according to claim 23,wherein said user self-reliant entity comprises: a biomechanical systemincluding a nanobot implanted in the brain, Micro-electromechanicalSystems (MEMS), or a synaptic chip implanted to function as abrain-to-computer interface and computer-to-brain interface for relayingat least one neural correlate of consciousness and other data betweenbiological and computerized neural networks within a bio-mechatronicself-reliant entity.
 29. The self-reliant entity according to claim 23,wherein said user self-reliant entity comprises: (a) a self-sufficiententity with superintelligence-like abilities beyond what a normal humanbeing can perform that includes at least one of the following: i. theability to selectively record and store in detail sensory informationderived by said entity along with said entity's perceptions of thoseexperiences; ii. substantially 360-degree audio-visual sensingcapability facing outward from the entity; iii. the ability to haveagents operate like self; iv. the ability to process new information andprevious information from said entity's logged database and derivedneural correlates of consciousness (NCC) database in a computer at leastseven times faster than human cognition; v. the ability to overcomebiological memory storage limitations by transferring said entity'slogged database and derived neural correlates of consciousness (NCC)database to a mechanical system; vi. the ability to overcome mechanicallimitations by transferring said entity's logged database and derivedneural correlates of consciousness (NCC) database to a biologicalsystem; vii. the ability to transfer said entity's logged database andderived neural correlates of consciousness (NCC) database to abio-mechatronic system to overcome purely biological or mechatroniclimitations; viii. the ability for a biological, bio-mechatronic, andmechatronic entity to communicate entity perceptions, a logged database,or a derived neural correlates of consciousness (NCC) database via abrain-to-computer and computer-to-brain interface; (b) saidself-sufficient entity having the ability to update its own computercode; (c) said self-sufficient entity having the ability to change itsappearance but retain the same internal knowledge; (d) saidself-sufficient entity having the ability to work 24 hours per day; and(e) said self-sufficient entity having the ability to surviveindefinitely.
 30. The self-reliant entity according to claim 23, furthercomprising at least one of: a. a cognitive memory, b. an artificialauto-associative neural network, c. a deep learning artificial neuralnetwork system, d. a pattern recognition system, e. a machine learningsystem, f. a predictive algorithm, g. a recursive algorithm, h. afeedforward network, i. a convolutional network, j. a self-organizingnetwork, k. a speech recognition system, l. a voice synthesis system, m.a machine translation system, n. an image to language translationsystem, o. a long/short term memory network, p. a multi-layered network,q. a supervised or unsupervised network, r. a network that includes andadvances off of a single activation neuron that sums up what wouldnormally be represented by processing an association of neuronsrepresenting at what one neuron encapsulates, s. a non-network computeralgorithm, routine, and model that operates to address shortcomings of aneural network; and/or t. a multi-class sensory signature segmentationsystem with labeling that yields natural language image and sentenceconstructs to process sensed data from the user and surroundingenvironment; wherein said host or recipient resulting mapping andmerging of sensed and processed data derive information that is learnedby said entity and define stored memory and neural correlates ofconsciousness that uniquely define said entity's self-perception andentity's perception of said surrounding environment.
 31. Theself-reliant entity according to claim 23, wherein said user conveyablesystem comprises: at least one of an imaging, auditory, olfactory,haptic, tasting, and cognitive sensing system with recording andfeedback components for affecting said entity's self or a recipientsystem.
 32. A method of logging and correlating experience data derivedfrom a user system and communicating said data to a recipientbiological, bio-mechatronic or mechatronic system, the method comprisingthe steps of: providing a user conveyable computer system withsubsystems comprising at least one of: (a) a biological neural networksubsystem; and/or (b) an artificial neural network subsystem; providinga surrounding environment sensing subsystem; providing a user portableelectronic device including an interactive graphical user interface andconfigured to operate as a host computer processing subsystem forcommand, control, and processing of signals to and from said biologicalneural network subsystem and/or the artificial neural network subsystem,and the surrounding environment sensing subsystems; providing sensing,collecting, recording storing, measuring and correlating experience dataderived from neural activity from said biological neural networksubsystem and/or the artificial neural network subsystem, and thesurrounding environment sensing subsystems; quantifying experiences ofthe user based on neural activity correlated to the surroundingenvironment and between the biological neural network subsystem and/orthe artificial neural network subsystem sensing, collecting, storage,measurement, and correlation computer subsystem; transmitting saidmeasurement data to said surrounding environment sensing, recording,collecting, and correlation portions of the computer subsystem andcreating relationships between said data corresponding to at least oneof: (a) said neural activity of said user, and (b) said datacorresponding to said user events and surrounding environment, andbetween: (a) biological neural network activity, and (b) artificialneural network activity, determining an extent of neural relationshipsbetween data received from said user portable electronic device and saidartificial neural activity sensing, surrounding environment sensing,recording, measurement, and correlation subsystems; determining whetherto keep or disregard said correlated data based upon at least one of:(a) pre-established rule sets, (b) thresholds, (c) vectors, and (d) userinteractive commands and control from said user portable electronicdevice; and communicating at least a portion of said correlated datainto at least one of a said recipient biological, bio-mechatronic ormechatronic system.
 33. The method according to claim 32, furthercomprising the steps of: a) initiating an independent, self-reliant,conveyable, computer system with computer subsystems and devicesoperated in real time, to dynamically correlate neural network activitydata with surrounding environment data; b) operating a first computersubsystem that includes and is compatible with said conveyable computersystem that derives, includes, and is compatible with a neuralcorrelates of consciousness database stored as non-volatile memory insaid computer subsystem that defines a taxonomy for an entity'sperception of self and the environment; c) operating a second computersubsystem including and compatible with said neural correlates ofconsciousness database operated upon by at least one biological neuralnetwork and artificial neural network that includes backpropagationprocesses that iteratively derive logic based outcomes that improveresults or achieve desired outcomes that are decided by said entity anddetermine said entity's activity; d) operating a third computersubsystem including and compatible with said entity's sensor arrangementto record and store in non-volatile memory entity neural sensingsubsystem and surrounding environment sensing subsystem signatures toformulate a plan, and acting with the intent to reach a goal based onthe plan derived and shaped by said entity's overall design; e)operating a fourth computer subsystem that includes an actuator andmanipulator subsystem that functions with the mobility and dexterityakin to a human; f) operating a fifth computer subsystem that includes arechargeable energy generation system, wherein the conveyable computersystem and subsystems, actuator and manipulator subsystems are incommunicating relationship and, once initiated, all operate as acohesive system akin to a human; g) said biological, bio-mechatronic, ormechatronic system's perceptions and existence being recursivelytransferred between a biological, bio-mechatronic, or mechatronic systemif said biological, bio-mechatronic, or mechatronic system so chooses.34. The method according to claim 32, comprising the steps of: providinga micro-electromechanical system (MEMS) in a computing system andnetwork as an interaction mechanism configured to deliver a fast fusionsensor system interfacing with at least one of: a) a neural activitysensing, measurement computer, b) surrounding environment sensing,recording, and correlation subsystems, or c) a mobile electronic devicewith at least one MEMS component, wherein, for at least one entity in aknowledge based computing system configured to store complex structuraland unstructured information, the sensor fusion interface is beingconfigured to generate a profile comprising entity state data, exposingthe profile to the computer processing device, and instructing aknowledge-based data component to update the profile in response to saiduser activity and said environment activity, wherein the knowledge-baseddata component is configured to integrate multiple sensor input datafrom a plurality of sensor modules according to a schema, and toidentify the entity and entity activity based on the multiple sensorinput data; said MEMS operating within at least one of said userconveyable system, peripheral or remote subsystem, or a recipientbiological, bio-mechatronic or mechatronic system.
 35. The method ofclaim 32 further comprising the steps of: providing a first collectionof neural activity data and metadata representing relationships betweenphysiological data and said surrounding environment data that includesrepresentations of brain activity unique to at least one user; saidfirst collection of neural activity data and metadata operated upon andderived by at least one of a computer incorporating an artificial neuralnetwork, a cognitive assistant that learns and organizes, and/or aquantum computer network; providing a second collection of data andmetadata resulting from operating on said first collection of data plusoperating on data and metadata derived from Internet queries of a mobilelife-logging system command and control subsystem with Internetcommunication query and search capabilities; said data from Internetqueries operated upon and correlated with physiological and surroundingenvironment data from said mobile data collection system to arrive atrelationships between said data; said data operated on and correlated byat least one of a computer incorporating an artificial neural network, acognitive assistant that learns and organizes, and a quantum computer;providing a third collection of data and metadata derived from saidfirst and second collection of data and metadata wherein the thirdcollection of data and metadata comprise interbrain data and metadatagathered by a mobile data collection system; said third collection ofdata and metadata representing relationships between the user's neuronsand components that characterize the minimum set of neuronal events andmechanisms sufficient for a specific conscious precept that forms newneurons and new neural correlates of consciousness that potentiallyrepresent novel, useful, and unobvious constructs formulated in the mindof the user; said data operated on and correlated by at least one of acomputer incorporating an artificial neural network, a cognitiveassistant the learns and organizes, and a quantum computer; andproviding a fourth collection of data and metadata resulting fromoperating on said third collection of data wherein computer processingincludes searching and comparing new interbrain neural constructsresulting in new correlations of consciousness with prior art constructsdetermining if the new neural constructs represent novel, useful, andunobvious art; said constructs being novel if not already discovered byother users and evidenced by prior art; said data operated on andcorrelated by at least one of a computer incorporating an artificialneural network, a cognitive assistant that learns and organizes, and aquantum computer.
 36. The method according to claim 32, furthercomprising the steps of: providing stored and derived data from saiduser conveyable system and said recipient systems; operating upon thedata to diagnose and treat an illness of biological condition of atleast one of the following: a memory loss, Alzheimer's, post-traumaticstress disorder (PTSD), attention deficit hyperactivity disorder (ADHD),death; and operating upon the data to diagnose and address a mechanicalcondition of at least one of the following: a degradation, disruption,breakage, failure, or computer system memory loss.